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error-prone areas, while proceeding with an overall design strategy; 
an emphasis on both manual and automated QC measurement, enhancing 
existing databases arid procedures; and an emphasis during the first \* 
year of the quality improvement program on the .development of 'the '. » 
measurement component of the program. The design for the OSFA quality' 
improvement program is discussed, witlf attention to* .opportunities for 
quality improvement in the current delivery system. Also cpnsidered . 
is the overall strategy for the improvement program that concentrates 
on institutional QC, external QC, internal OSFA QC, and a QC 
management information system. The critical targets for internal 
quality improvement are identified and measures are proposed. 
Finally, an implementation plan f^pr -the improvement program 
emphasizes the role of the corrective action component, implementing 
corrective action using the critical activities, and priorities for 
ongoing quality improvement (SW) 
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PREFACE 



The Office of Student Financial Assistance (OSFA) of the Department of 
Education has contracted with Advanced Technology, Inc., of McLean Virginia, and 
its subcontractor, Westat, Inc., of Rockvilie,* Maryland, to conduct a three-year 
quality control project-(Contract' No.: 300-80-09,52). The focus of the project is the 
Pell Grant Program, the largest of the student grant programs administered by 
OSFA. The objective of Stage Two (Part One) of the project is to design a quality 
control system to' measure and- analyze program performance. The reports 
completed" to date under Stage Two (Part One) include: 



Quality Control (QC) System Development for 

the Pell Grant Program: A Conceptual Framework 

Action Plan for Quality Control System Design: 
A Working Paper I 

* . ' » '* 

A Comparison of Title IV Student Assistance 
Delivery Systems 

Preliminary Quality Control System Design . 
for the Pell Grant Program 

A Framework for a Quality Control System 
for Vendor/Processor Contracts 

Recommendations for Improving Quality in 

the Campus- Based Program: FISAP Process •» 

TtaSp^icationsior Conducting 1 . '. 
an Annual Assessment of Overall Payment - 
Error in the Pell Grant Program 

* 

Technical Specifications for QC System 

Enhancements to the Manual GSL Interest 
jBilHng Process 

Corrective Action Framework for the Office 
of Student Financial Assistance 

Quality Control Procedures Manual for Manually 
Processed Interest Payments Guaranteed 
Student Loan Program 

Quality Assurance for Vendor/Processor Contracts 

OSFA Goals and Objectives System Strategies for 
Improved Program Planning- and Management 

Office -of Student Financial Aid Quality Improvement 
Program: Design and Implementation 
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EXECUTIVE SUMMARY 
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During the past two years, the Office of Student Financial Assistance has been 
^engaged in an effort to develop an internal quality improvement program. The 
purpose of the. program is to develop a systematic approach to identifying, 
measuring, and correcting errors or tendencies toward .errors in the student aid 
delivery system. The design for tru> program includes both a measurement 
component and a corrective action component. This report describes the purpose 
arid direction- of the OSFA' quality improvement program. The report Has four 
* chapters. v ♦ - ' 



Chapter i provides background on the quality improvement program. Some of N 
the basic points covered include: * • ' 

■ t ' ' • '-'.,"> 

A review of the literature suggests quality control and corrective action 
can be most effectjve when targeted at the most error-prone areas; , 

• The ^verairOSF A quality improvement program has QC measurement 
and corrective action components, both designed to target technical 
i assistance at hig^error^rone areas. ( j 




Chapter 2 Yeviews the general approach to quality improvement dexelope6 
during the Stage H Pell Grant Quality Control study. This approach consists of the 
following key elements: ' * - ■ 



\ 



An overall strategic approach that enables OSFA to target technical 
assistance on high error-prone areas, while .proceeding with -an overall 
design strategy? ' * . V 

An emphasis on both manual 'and automated quality control measure- 
ment, enhancing existing data bases and procedures^ to the extent 
possible? 

An emphasis during the first year of the quality improvement program on 
the development ofr-the measurement component of me program. 



( 



i 



ERIC 




IV 



Chapter 3 'presents the overall design for the OSFA quality improvement 
program.' First, the opportunities for quality improvement in the current delivery, 
system are reviewed. Marginal improvements to critical activities in the current 
system can alleviate some of the most seriously negative effects of the current 
system, including deficiencies in fund control and availability of program 
information.. i 

Second, an overall strategy for the OSFA quality improvement program is 
presented. OSFA is undertaking a quality improvement program that concentrates 
on: 

- • M ■ ' . ' ' / 

• Institutional Quality Control, which is encouraged by Federal regulations 
* • and facilitated by institutional quality control guidelines; 

• External Quality Control, which can provide OSFA with an ongoing 
,j measurement of%Verall error rates for all programs; 

, h 1 Internal OSFA Quality Control, -which includes a supplemental, effort* to 
identify error prpne' activities, develop measurement mechanisms for 
* -these activities^and identify corrective action options; 

' A Quality Control Management Information Systems (QCMIS), that will 

■ * ' ^ provide an overall QC report irtg and information system. w 

Third, the optical targets for interna] quality improvement are identified and 
measures are proposed. The critical activities, some of which were addressed during 
Stage Ihof the^ Pel* QC Study, provide OSFA tfith a road map for future internal 
quality improvement. 

i \ Fourth, the next steps in the OSFA* quality improvement process are 
considered. „ These include: : 



• Selection*of new targets for technical assistance from the list of initial 
^activities; , . 



• Development of a "systematic approach to quality ^mj^povement for 
critical activities, which is addressed in the final chapter; 

• Development of tHe QCMIS framework which would provide OSFA with a 
well-defined QC information and reporting system* " 



apter U presents an implementation* plan for the OSlFA quality improvement 
. This chapter includes: * • 

An overview of the entire OSFA quality improvement program, with a 
special emphasis on the role of the corrective action component; 

An implementation plan for the program? 

A strategy for implementing the corrective action component using the 
Critical activities identified in Chapter 3; 

' Priorities for ongoing, quality improvement. 
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CHARTER 1 




' 'BACKGROUND 



This report presents, the results of, the. Quality Control (QC) System D 
task, Stage Two of the Pell Grant Quality Control Study. This task was initiated in 
an attempt- to design a QC system for the Pell' Grant Program. - Based on the 
preliminary design of theiPell system and a comparison of the major' student aid 
- programs, the scope of the task was broadened to include the Guaranteed Student 
. Loan (G§L) andl the Campus-Based (CB) programs, and to provide technical assist-' 
ance' # in qualitative improvements in the delivery of , these major- student aid 
programs operated by the" Office ol Student Financial < Assistance (OSFA). This 
refined approach was based orr the understanding that there were many basic 
/ similarities In the delivery systems for these programs and that quality Improvement 
s was the responsibility of functional units within OSFfr. The quality improvement 
program discussed in tbfe paper can provide OSFA with a* systematic process for 
identifying and correcting error^prone ^oints^in the delivery system. A major 
outcome of this design and technical assistance* activity is a plan foi^an ongoing 
quality improvement program .described in this final jpeport.* This chapter reviews 
background and context for the design sttioV and discusses the m*eaning of 
quality control. /y 

t * i 

THE PELL GRANT QUALITY CONTROL STUDJ^ « 

Quality, while often considered ill-defined in many organizations, does not just 

■ . * 

• happen. It can be realized only through management programs that better utilize 
personnel and systems to improve prbduct development and. delivery. In a lar§e 
social program, such as the Pell Grant "Program and 'other student aid programs, 
millions of people are directly affect eVby the quality of the delivery system/ The 

»,-4rftimate test of quality for a social prpgram is whether the beneficiaries of the 
program receive the correct 'amount of aid, on time, ana with a minimum of error. 
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Additionally, from the perspective of the Federal government, it is important that ■ 
aid be delivered in as effective and efficient a manner as humanly, or technically^ 
possible. ^ These factors—quality of the product and delivery system— are, of concern 
to postsecondary institutions, state agencies. and to the Federal government. 

The Pell Grant Quality Control Study was designed to provide^OSFA with a 
comprehensive examination of the status of the current delivery system and to 
incorporate basic quality improvement principles. The quality control literature 
revie^ discusseji below provides three fundamental principles for quality 
improvement— prevention, identification, and elimination (PIE). These are generally 
characterized as follows 

« y 

• Prevention refers to the design component of any production of delivery 
system. Prevention of error must be considered when a delivery process 
is designed originally, or in subsequent redesign; 

• Identification of errpr becomes important once a production or delivery 
. system has been implemented. Typically, this*wouW involve a statistical 

sample of products (awardees in the case of student aid) and the 
establishment of measures and standards of routine error measurement; 

. Elimination refers to the process of correcting error in a production or 
delivery system once it has been implemented. This process is usually 
referred to as corrective action. . * 

The three-stage. Pell Grant Quality Control Study, illustrated in -Figure 1-1, 
was a comprehensive quality improvement project. Stage One was a study of the 
quality of the current delivery system for Pell. Stage Two^oflhe project was^ 
divided into three parts: Part- I was targeted at designing and jmplementlng an 
ongoing measurement system for quality control in OSFA, Part II was an assessment 
of the effects of the current and alternative delivery system design (currently 
underway), and Part III was an analysis of Stage One data. Stage Three, currently 
underway, provides a restudy of quality in the delivery .system. 1 

The three components of Stage Two closely parallel the 'PIE concept. Part II, 
Delivery System Assessment, focused on prevention by addressing basic long-range 
design problems with the student aid delivery system. .Part ffi, Follow-on Analysts, 
is focused on continued and refined identification of errors in the delivery system. 
Part I, the QC System Design Component, focused on the elimination of error in the 
current delivery system through the implementation of *an, ongoing quality 
"improvement program. ^ % « 

't 0 



STAGE O NE 
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STUDY OF QUALITY IN PEL'L GRANT DELIVERY .SYSTEM 
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STAGE TWO 
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STAGE THREE 



PART It RESTUDY OF QUALITY IN PELL 
GRANT DELIVERY SYSTEM* 



PART 0: ONGOING QUALITY CONTROL 





PART I: 
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DESIGN AND IMPLEMENTATION OF 
OSF A QUALITY CONTROL 
IMPROVEMENT PRQGRAM 
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PART tk 


DELIVERY SYSTEM ASSESSMENT 
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PART HI: 


FOLLOW-ON, ANALYSIS 
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FIGURE 1-1 
PELL GRANT QUALITY CONTROL STUDY 
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This is the final report for Stage Two, Part I of the Pell Grant Quality 'Control A 
Study. As tj)e figure illustrates, it draws 'from parts of Stage One and otter parts of 
Stage Two* and provides a framework for continuing the OSFA quality improvement 
program. » ¥ '- ' 

The findings of Stage One had an important impact on Shaping Stage Two of 

the study. Stage One. Indicated substantial dollar errdr in awards to students during 

the 1980-81 academy year; •' Dollar error -was defined as* the actual award 

-disbursements as listed' in records at the sampled undergraduate institutions in 

Spring, 1981, minus what Advanced Technology calculated the correct disbursement 

to-be using the best available information on application data* cost of attendance, 

and enrollment status. Total dollar error for FY 19&1 was estimated, to be $275 per 

recipient, or $650 million of the $2.2 billion (a 30 percent error rate) awarded to the 

2.36 million recipients represented by the sample. An estimated?} percent of the 

recipients* received an incorrect award, although in some cases the amount of 

incorrect award was quite small. Approximately^ percent of recipients had award 

errors in excess of $150. Net error (ovtfrawards minus undera wards) was $402 

million. Fifty percent of program recipients (of approximately 1.2 mUlioft students) 

received overawards totaling $526 million. Another/ 21 percent of recipients 

(approximately .5 million students) received undera wards totaling $124 million. 

' Stage Two moved beyond these basic research findings and was broader than 

Stage One in two important ways. First, the scope of work actually included 

providing technical assistance to operating units in OSFA. The technical assistance 

was provided, on a priority * basis, to operating units responsible for quality 

improvement activities. This was a logical extension of the initial design since the 

identification of areas in need of technical assistance was baseitupon an evaluation 

of the delivery system- performed during Stage One of .the study. The technical 

assistance activity -actually took the form of assessing needs for corrective action 

■* * , 

and making recommendations for specific corrective actions. 

Second, Stage Two, especially Parts I and II, included other Title IV student , 

assistance programs; particularly the Campus-Based Programs— the Supplemental 

Educational Opportunity " Grant (SEOG), National Directs Student Loan (NDSL), 

e Work-Study Program* (CW-S)— and the . Guaranteed Student Loan Program 

The GSLP includes both Guaranteed 'Student Loans (GSLs) and Federally 

Student Loans (FfSLs). . 
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THE WANING OF QUALITY CONTROL 




/ • .■ . • - - . ■ ■ 

American government and industry have recently rediscovered the meaning qi . 
-quality improvement and quality control. • During the past decade, as problems 
emerged in tf*e U.S. economy, government and industrial leaders began to ask basic 
questions about the qulfclty of"producti<Jri in the industrial sector and the quality of 
service delivery in government- • . I . % ' . 

In the e^rly 1970s, the Federal government faced the ' monumental task of 
going into production with* a massive entitlement- program; the -Basic Educational 
Opportunity Grant Program, within a one year time-frame. This task was accom-s 
piished successfully. Each subsequent year, me program* grew in size and changed in 
* some basic programmatic wayr consequently, the actual delivery of grants remained 
the critical issue. There wa® little time to concentrate on the quality of the system 
»d for delivery. The other student aid .programs have faced similar probiemsjptth 
need to deliver a program that is continually being changed in the legislative ' 
* process. For example, the G$L program has undergone major programmatic changes^ 

* through reauthorizations and technical amendments of the Higher Education Act 

that have resulted in nearly constant modifications of the GSL delivery system 
during the past 12 years. Consequently, there is* need for a systematic quality 
improvement program for thetna)or student aid progpamsr • • . : : • 

. The QC literature provides a framework for a systematic quality improvement 
program. 3u*an and Gryna, 1 two leaders in the QC field, ctefine quality as fitness 
for use. Accordingly, QC should include activities which assure that products or 
services are fit. 3uran and Gryna emphasize that quality activities encompass the 
life of a product, from design to post-sale, although they recognize that^often only a . 
limited, range of these activities can be provided at any one time. For student aid 
programs, thlg perspective-suggests that quality control should encompass the entire 
delivery process, from the application planning for student aid programs, to 
reconciling accounts after the aid is delivered. An important distinction can be 
made between sporadic defects and chronic problems. Wea|Jy, the QC process 
/ should involve a breakthrough process to eliminate chronic problems, while sporadic 
J! errors can usually be eliminated through preventive QC. Figure 1-2 Ulustf^es the 
"' differences in the two approaches to QG. 




^orah* 3.M. and Gryna, FJfc.rQuallty Planning and Analysis ,- New York* McGraw- 



Hill, 1970. ' ^L:>-\J% r ^ „ • 
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QC WITH EMPHASIS ON CONTROL 



QC WITH EMPHASIS ON BREAKTHROUGH 



I. 
2. 
3. 

.-5* 
6. 



8. 



Choosing the control subject— defining the quality, 
Characteristic, or effort that must be regulated . 

Choosing a unit of measure— defining the terms in 
which the control subject will be measured 

Choosing a standard—defining the desired level of 
performance for the control subject 



Designing a sensor—creating a method of measuring 
the control subject ^ . * ; 

Measuring performance—performing the actual 
measurement ' . 

\ 

Interpreting results— comparing the actual measure- 
ment,to the standard 

Decision making— deciding on the action, if any, to be 
taken to the standard 



Action— faking the specific steps to bring performance 
up to the^standard , 



Breakthrough in attitudes— convincing those responsible 
that a change in quality level is desirable and feasible 

Discovery of the vital few problems— determining which 
quality problem areas are most important 

Organizing for breakthrough in knowledge— defining the 
organizational mechanisms for obtaining the knowledge 
for achieving a breakthrough * « 

Creation of thesteeririg arm— defining and staffing a 
mechanism for directing the Investigation 

Creation of the diagnostic arm— defining and staffing a 
mechanism for executing, the technical investigation 

Diagnosis— collecting and analyzing the facts required 
and recommending' the acllon needed 

Breakthrough in a cultural oattern^determining the effect 
of proposed changes on the people involved and finding ways 
to overcome the resistance to change . - 

Breakthrough in performance— obtaining agreement to take 
action 



9. Transition to the new level— implementing change 



i. 



2. 



3. 



5. 



6. 



7. 



Source: 



Duran, 3M. and Gryna, F.M., Quality Planning and Analysis, New York: McGraw-Hill, 1970. 
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FIGURE 1-2 
BREAKTHROUGH vs. CONTROL 
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Historically, QC in the United States has emphasized technical approaches 
such as statistical QC, reliability, and product assurance. Increasingly, however, it 
Is important for organizations to, establish quality policy, with specific quality 
objectives that should be conveyed in written form. In his 'most recent work, 3uran^ 
argues that two types of QC are necessary, managerial and technical. In building a 
management commitment to QC7a successful QC analysis must consider the 
existing organizational responsibilities and the new design should be built around 
them. Additionally* a top management commitment to, quality improvement is 
necessary to maintain an ongoing quality improvement program. 

, A variety of well-established tools and techniques are available for "jtJC. The 
American Society for Quality Control maintains a publications p^igram that covers 
such topics as national standards^ and guidelines lor managing vendors.* There are 
well-established guidelines for quality audits and quality cost analysis,? and wetf- 
defined'plartfc for e*stablishing quality improvement programs. 6 

One, especially useful analytic technique in quality analysis is the Pareto 
principle wmch states that losses are never uniformly distributed over causes 
(quality or characteristics). Instead, losses are always unevenly distributed so that a 
small percentage, or a vital few of the causes, contribute, a higher percentage of the 
loss of error. This principle can be used to analyze the distribution of loss due to 
error. The results of this approach are often a boon to managers, and others. 
concerned with instituting QC; it c^ facilitate ato economical, targeted^attack on 
the bulk of quality losses* Ouran and Gryna (1970) argue that this simple truth, the 
Pareto principle, makes a quality improvement program possible. Once the vital 
few problems have been identified the^ can be targeted for management of 
systerlatic processes, identification- of high error-prone areas, then targeting or 



2 3uran, 3.M., Course on Management of Quality, New York: 3.M. 3uran, 1981. 

^American Society for Quality Control, American National Standard: Definitions, 
Formulas, and Tables for Control Charts , Milwaukee, Wisconsin; ASQC. 1978. 

■ • 

• 

* American Society for Quality Control, Guide fe# Managing Vendor Quality Costs , 
Milwaukee, Wisconsin: ASQC, 1980* 

■^Bajaria, Hans 3. (Ed.), Quality Assurance; Methods, Motivations and - Management , 
Dearborn, Michigan? Society of Manufacturing Engineers, 1981. f ' ' ~ 

6 Kidweil/ 3.L., A Profit Plan for Quality, Waterford, Connecticut* 3ohn L. Kidweli 
Co.,197^. ^ 

fc 
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technical assistance. These areas can be the basis of a systematic quality 
improvement program. 7 J 

9 These concepts are useful for student aid programs? however, the literature on 
QC is dominated by the private sector, especially industrial ^production. "Most 
government standards have been generated by and for defenseSoritractors. Conse- 
quently, the decision, to institute the quality improvement programs for OSFA has 
been a one-of-an^d effort: it has applied basic principles developed in- |he <^C 
field to the specific and unique problems of student aid delivery. <f 

THE OSFA QLJALrTY IMPROVEMENT PROGRAM 

' The overall goal of this task is to establish a framework for improving the 
quality of OSFA programs. A significant way to raise program quality is to 
routinize the process of QC. QC, as used in this project, is the process of 
preventing, identifying, and eliminating Sources of error in a delivery w production 
system. The concept is most comprehensible when used in an industrial setting. For 
example, it is ea*sy to see the need for QC measures in the production of 
automobiles. If the quality of a particular make' of automobile varied greatly from 
car td car, the result, would, be expensive corrective action costs to the manure- 
turer and decreased sales to • consumers. Quality control somewhat more 
ambiguous in a social service setting, such as. the provision of student aid? however, 
it is no less important. The Pell Grant Program, one of the largest student, aid 
programs, illustrates this point. The annual overpayments in the Peii program, due 
fo various institutional, student, and processor errors, are estimated in excess of 
$^00 million. This amount provides a great strain on program resources and, due to 
anndal funding ceilings, may reduce the size of the average award at, the same time 
that education costs continue to rise. ^ 

In both industrial and social service settings, QC .measures can increase the 
efficiency and effectiveness of the delivery system by reducing costly errors and 
raising the quality of services or products. In order to integrate QC into the 
everyday operations of an organization, at a minimum: 

• Output standards must be established? • 

• The established standards must incorporate technical specifications 
prescribed by the provider of the service or product and expected by the 
recipient or consumer; 

7 Kidwell. ibid. 
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• A system for monitoring standards and correcting errors must be made 
. operational. - * 

Among the possible operational procedures in a student aid specific QC program are 
the following: 

Develop standards and measures for monitoring the delivery of student 
aid; 

I . , . . • 

Measure performance of student aid delivery against specified measures; 

Determine and monitor errors in eligibility determination and award 
processing; 

Identify sources and probable causes of errors to plan corrective actions; 

Develop corrective action procedures as an integral part of the process- 
ing functions; . 

Develop standards and measures for monitoring the results of corrective 
actions; ",•/.••* 

Ensure that various actors (e.g., processors or institutions) are operating 
in accordance with specified procedures, regulations, and standards; 

Report appropriate QC information to Department^ of Education 
personnel on a timely basis. * -: 



In order to improve the quality of OSFA programs, two distinct procedures 
must be developed. First,. £n ongoing structure or framework to determine sources 
of program error and to measure it must be developed. This component of a quality 
improvement program may be called the' technical measurement component. 
Second, a formal mechanism for designing and .selecting procedures to eliminate 
existing error must be designed. This component of a quality improvement program 
is called the corrective action component. 

When these two components of a quality improvement program are integrated, 
they become a process for maintaining QC When viewed in the context of a quality 
control process, each component has a series of well-defined subcomponents or 
steps. The technical measurement component includes the following steps: ' 

• . Define the sampled subject for control; 

• * Define a unit of measure; 

• Establish a standard of performance; , ^ 
m Create a measuring device or procedure. 
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The steps in the corrective action component ares * 

G 



Mobilize for measurement; „ . 
Compare actual performance with established standards? 
Make management decisions on type ol corrective action needed; 



" • 



Implement^cprr ecti ve action* 



An illustration of tHe interrelationships between these steps in a well-integrated QC 
process is shown In Figure 1^ The QC^process is illustrated as a cyclical procedure 
since the process is ongoing/ % 

The *QC cycle was used as a basis for the design of the OSPA quality 
improvement program* The emphasis of this taste was on the design of a technical or 
measurement componehi for the Pell Grant Program* The overall design for the 
OSFA Quality Improvement Program includes both measurement .and corrective 
action components. - \ — ¥ 
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System 
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STEPS IN TltE DEVELOFMENT OF AN OSFA 
QUALITY IMPROVrmfT PROGRAM 




CHXPTER2 . \ 
GENERAL APPROACH 

\ 

... j 1 f* * 

This chapter reviews the general approach use^in the design of the Quality 
r Improvement Program. This program provides OSF A with a systematic methodology . 
for identifying, measuring, and,correcting errors in the student aid delivery system. 
The chapter has three sections. The first, reviews generic approaches to quality 
improvement, including the strategic approach which was used for the task. Sec 
alternative QC system configurations for the measurement component of the OSFA 
QC system are reviewed* Finally, the actual framework used in the task is 
summarized. The general approach presented in this chapter is a tested metrol- 
ogy for improving quality in student aid program's* $> \ « 

GENERIC APPROACHES TO DEVXLOPHJ^QUXUlfif CONTROL PROCEDURES 

jLm . ■ • * 
. W* .•••'*' • •> 

The quality control process should not merely be an afterthought or an 
addendum to an existing system. Unfortunately, the provision of student aid does 
not represent this ideal situation* Although some concern has been paid to quality 
control procedures, no rigorous and methodical quality control process paralleled ihe 
development of the delivery system. Therefore, quality Control procedures must 
subsequently be integrated irfto\n operational delivery system. Three generic 
approaches have been identified for introducing quality control procedures into an 
existing delivery system. The first is characterized, as the incremental bottom-up 
approach, the second as the comprehensive approach, arftJ 4he third as the strategic 

or modular approach. \f~ - -* 

* The incre men t a l bottom-up approach to quality control development typifies 
the approach used in most government agencies, educational institutions, and 
industrial settings. This approach assumes that functional subuhits within an 
organization have responsibility for their own corrective actions and as a ^resGJt, no 
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system-wide quality control plan exists. Therefore, quality control development is 
usually sporadic, incremental, aijd seldom the. outcome of coherent planning. 
Further, since corrective actions are identified by subunits, quality* control problems 
^Involving interface with other subunits are rarely resolved. The typical steps* 
followed by an'organ^ation taking an incremental approach to quality control ares 

• Functional subujtits (division,* branch, etc.) discover error-prone areas 
through problems with 'the system in operation; 

\ • Ptens foj corrective actions (new procedures or system changes) are 
' ftevelopjB and based on needs as they arise— usually a limited range of 
optiowr^e considered; 

• Managers seek new resources for corrective actions— system develop* 
* , -ment or implementation of .new procedures— but problems that involve an 

interface, with other subunits are often Ignored unless the* resource issues 
can be resolved; / • . » 

* • • 

• . Quality Control procedures are developed and implemented only as time 
' and resources permit. . . . * ^\ 

■ - - v - , ■ 

*w .The incremental approach has dominated quality control development in 

•student^ aid. While the Division of Qualify Assurance (DQA) has implemented some 

quality control 1 reporting requirements m |be past* there have been no systematic 

attempts to develop quality control procedures. An incremental approach to quality 

control is likely to produce some serious problems. These includes 

\3 • 
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Completion of specified tasks inMhe necessary time frame without 
concern for smooth operation and reduction of error in the system and 
efficient use of personnel and other resourcesf ^ 

Inconsistency ancr^variation in the ways' functional subunits' deal with 



quality control problems; 

r * i' 



• M Acfte^ problems in the quality control of products that cut across 
f unctionarstibunits (or that involve more than one Division/Branch). 

The comprehensive approach assumes that anything that can go wrong with the 
delivery system will go wrong; therefore, it is important to identify every possible 
error in the system and design corrective procedures. In order to intsoduce quality 
control procedures into the student aid delivery system according ti this method- - 
ology, it would be necessary to identify all program subsystems and all the major 

( 
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actors or components in each subsystem and then define a responsive series of 
corrective actions. Specifically, the basic steps in a comprehensive approach to 
quality control are: » ■ . ^ » , 



it Identify the major subsystems of the programs (this includes pre- 
application, application, eligibility determination, benefit calculation, 
fund disbursement, and account reconciliation); 

► Identify the major actors or components for* each subsystem (for 
example, for the Pell eligibility subsystem, actors include students/ 
parents, institutions, and application processors); 

» For each actor in each subsystem, identify acceptance measures of 
possible errors in the system; 

• - '? \ 

► Define measures for each set of standards including identification of 
data elements and procedures for information collection; 

t Determine ' the components « of each 'subsystem thatf merit development 
and inclusion in the quality control system and evaluate the feasibility of 
including the various information^ sources in a quality control data base; 

• Proceed with system development on the select subsystems (design and 
develop procedures for implementation of selected components of each 
subsystem). > «'.....*•• 

The strategijf approach assumes t£at the major sources of error in student aid 
programs can be identified and corrective action should be made in these areas 
according to a step-wise or modular methodology* Thus, to utilize thii approach, 
significant sources of error must first be identified, prioritized, and therrcorrected 
in a hierarchical manner. The basic steps in the strategic approach are: 

* < 

* • Conduct a functional analysis of the operating system, including infor- 
mation requirements, linkage structures, and breakdown points. In other 
words, identify the places in the system, where corrective action can be 
taken and monitor progress; 

• Identify significant sources of error in the program; 

► Select and prioritize targets for systematic "QC development and the 
time frame for specifications, design, development, and installation of 
each selected measurement systems; . 

► Proceed with systems development for selected measurement systems 
(e.g., develop procedures and systems manuals, usef manuaijiijitern 
specifications, and software specifications, as necessary); y^w* 




. • Perform system tests on selected measurement systems as they are 
completed, . v . 

Since the Stage One study identified major problem areas in the delivery of 
student^ aid, the project team and.OSf A decided that an incremental approach to 
quality control was not adequate. Bp|hr|he comprehensive and strategic approaches 
Jo quality control were viable stratM||||Sbwever.? * r 

For the Pell Grant Program, wfe^hiprehensive approach would result in an 
elaborate quality control system that imposed new data collection procedures on top 
of the already existing system. It would permit the development of a separate or 
Stand-alone Quality Control Management Iriformatidn System (QCMIS). It could also 
be used to produce quality control manuals for training ED personnel (Central and 
Regional) and institutional representatives in quality control procedures. To the 
extent that the system used automated data collection and analysis procedures, it 
would be labor intensive since virtually an entire new^set of procedures would be 
needed for each component of each subsystem. Also, sophisticated data base 
management procedures could be needed centrally, de pending on how much of the 
system was actually implemented. The comprehensive approach was considered too 
ambitious for implementation. Trying to implement the system all at one time 

A 

would put great stress on the entire delivery system. 

The strategic approach to quality control development in the Pell program 
would permit the incremental implementation and testing of modular quality control 
subsystems designed specifically to reduce errors in the system. The njioduiar 
approach could also permit the use of up-to-date electronic technology utilising 
existing data sources, where* appropriate, rather than developing entirely new data 
sources and reporting procedures* I? might also result in more systematic analysis 
and reporting on data currently collected. ' Some new data collection would 
invariably be required, however. The strategic approach provides OSF^with a 
flexible approach to targeting resources on areas of greatest need. The risk 
associated with this approach is that some important area could be overlooked, and 
consequently, a major problem could go unattended. 

After analyzing the two approaches the project team and DQA decided to use 
the strategic approach. The basic trade-off between the comprehensive and 
strategic approaches is comprehensiveness versus timeliness. Since there is an 
immediate need for quality control procedures in the student aid process and since 
funding is tight in all government programs, .the project team recommended utilizing 

' r ' ' . 



I 



( 



the strategic approach, to introduce quality control procedures.' This approach* has 
the greatest potential fori . * 



\ 



• •. Reducing errors in the Pell program; 

• ■ Adding, other 'Student aid programs to the quality control proems on an 
ongoing basis; .« •- - * - \ • ' 

• Pilot testing quality control components earlier in the study; 

• , Developing a sound responsive methodology to corrective actions^ 

• - - : < " " ' ' , 1 - " ' ' ■,' 

Alternative qc system^nfigurations in ospa - 

■ - . - : , . . - ' • • 

^Two important design issues concerning the QC system or program are* (I) the 
degree to which the sWem will be automated; and (2) the degree to which existing 
reporting and informatio\j>rocessing can be utilized. These issues, were considered 
early in the design process as part of the conceptualization of the quality 
improvement program. 

The issues of automation and report formats can be used to generate five 
feasible configuration options* * 

• ..Option 1: a QC system providing manual (enhancements to the existing 
delivery system and requiring new reporting formats; 

• Option 2: a combined manual/automated QC system requiring new 
reporting formats; 

• Option 3t a combined manual/automated QC system using existing 
reporting formats; 

• Option Mi a fully automated ' QC system using existing reporting 
^ formats; 

• Option 5: a fully automated QC system requiring new reporting 
formats. 

/ 

/ - 

V ■ 

Existing data are not of sufficient quantity or quality to make providing manual 
enhancements to th£ -existing system and using existing requirements a viable 
option. Each option can be compared using the following evaluation criteria: 

• Feasibility of the system design (Can it be done?); 
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Potential tor't e^ci^g error (Will H save money ?J; , 
% ^beyelopmeniai 'tests (How much?)* . * v n t 
# ../Interface with the delivery system (Will it work?). 

Option 1: Manual Enhancements V- \ 

h is possible to develop a quality control MIS that essen-fialiy provides manual 
enhancements: to the^ existing Pell delivery "system. This MIS will require -develop- 
ment of detailed report formats for each og^he major actors in the financial add 
delivery system.. The reports would be enteredjnto a- filing system in the Division of 
Quality Assurance and used as a means of monitoring and tracking progress on 
certain key quality control areas. Either the comprehensive or the "strategic 
approach to quality control system development could be used to develop the manual 
enhancements, although the comprehensive approach is easier to adapt to this 
option. The major problems with this option arAhat it would add to the reporting 
burden and introduce excessive time delays in program monitoring due to its 
nonautomated format* 

Option &~ Combined Manual/ Automated MIS Requiring New Reporting Formats 

This option would essentially take the approach in Option 1 and where 
appropriate, automate data files. Other files* including periodic summary reports 
rising aggregated data, would remain manual, this option has the potential for 
providing data on a somewhat more timely basis than Option 1 but would provide an^ 
additional layer of reporting on top of the existing delivery system. It cpuJa' use 
either the comprehensive or strategicjjystem development approach but would be , 
more adaptable to the former. c< 

Option 3s Combined Manual/ Automated MIS Using Existing Reporting Requirements 

This option would have seme of the same features as Option 2 but would 
emphasize new analyses of existing data sources rather than development of entirely 
new reporting formats. This approach would be flexible enough to add other student 
assistance programs as necessary. In this way, a series of QCM1S subsystems could 
be constructed that dealt with critical points in the delivery system. .This option 



V 



would be most adaptable to the strategic approach for quality control system - 
development, 

w 

* 

Option*; Fully Automated/Integrated MIS Using Existing Reporting Formats 

v _ , . . . '....\ * 

This option would take the approach in Option 3 to the fullest jjossible . degree 
of system automation* Such an option would .integrate the core QCMIS with systems 
that interface with the major actprs 1 operating systems. Monitoring, comparing 
results to specific standards, taking routine actions, and reporting could be buuVtnto J-'. 
the system. Such an' approach would be dependent on automation of most 
components of the student aid delivery system. This option could use a variation of 
either the "strategic or *he comprehensive \approach to systems development but ^ 
would have to be done in combination wit^ the redesign of the entire delivery 
system. '•' . ' , . « 

" . . I .... o 

Option 5: Fully Automated MIS Requiring New Reporting Formats 

' • ' " m < . ■ > * 

Option 5 is identical to Option * except existing data would not be considered 
sufficient to create a functional MIS. Thus, additional reporting burden would be 
added for actors in the delivery system. _ ' * 

m * 

Assessment 

-. * ■* 

Figure 2-1 summarizes the preliminary assessment of the five ♦generalized 
options against the evaluation criteria. On the basis of the preliminary assessment, 
it is possible to make an initial judgment about which QCMIS configuration option is 
most desirable. 

Option 1 would probably have relatively modest results on reducing error, 
moderate developmental costs, and would not create an integrated QCMIS delivery 
system. Option 2 would increase the potentiator reducing error but would raise ~ 
developmental costs. Option 3 has high potential for reducing error, would require 
moderate developmental costs compared to Options 2, 4, and 5, and would be 
partially integrated imp the 'delivery .system. Options k and 5, while having the 
highest potential for reducing error, do not appeat feasible at the present time. Of 
the five options, Option 3, a combined automated/manual system-using existing data 
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OPTION I 



Description , Manual QC System with 

J New l)a fa Sources. 



V caslbllily 



Potential for 
Reducing Error 
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useful.* 



|Lqwj> f Delay ibtrilt Into 
reporting Hmtt, rnofiltaf- 
Ing Iroftt-cnd Cdrr eettve 
actio*** tmpfciTicritatto* 
may not lead to error 
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Developmental 
Costs , 



Moderate. (Rc^cs 
devefepSfS entirely new 



Interface with 
Delivery System 



Not Migrated * (ttcsttlt 
parallel system.) 
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. OPTION I 



Combined Automated/ 
Manual QC System 
with New Data Sources. 

H{gh. (Can be Imple- 
mented with existing 
delivery system.) 



OPTION J 



Combined Automated/ 
Manual QC System 
with New Data Sources. 



High. (Can be Imple- 
mented with existing 
delivery system.) 



OPTION* 



Fully Automated, with 
Cxistmg Data Sources. 



Low. (Requires ma|or 
changes in delivery . 
system.) . 



OPTION > 



Fully Automated, 
Integrated QC System. 

* t 

Low/ (Requires delivery 
system reassign.) 



• Moderate. (Delays built 
mtd reporting limit 
monitoring front-end 
corrective action.) 



High. (Integrated 
approach permits mon> 
monitoring front-end 
corrective action.) „' 



High. .(Integrated 
approach permits mpnt- 
monitorlng front send 
corrective action.) 



High. (Integrated 
approach permits monitor- 
ing front-end eorreethre 
acthm.) 



High. (Requires devel- 
oping entlrely.new 
system,) 

Not Integrated . (Result 
b creation ot new, . 
parallel system.) 

- , /" 



Moderate (Uses exist- 
Ing data to fhe extent 

possible^ 

Partially Integrated. 
(Result \? Integration 
of QC subsystems with 
Pell delivery sjbtem.) 



High, (Requires auto- 
mating new data 
sources.) 



Partially Integrated. 
(Result Is Integration 
Of QC subsystems with 
Pell delivery system.) 



4 

Unknown. (Inclined In 
delivery system redesign 
which would-be cosily.) 



Ful l y Integrated . (Result 
Is fall Integration olQC 
into Pell delivery systepv) 
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sources, appears to be the most desirable and viable QC system configuration. It* 
would create a QC system that is resource saving, using the enhancing existing data 
and management practices, rather than resource draining, placing new demands on 
the current delivery system. 

FRAMEWORK* FOR DEVELOPING QUAI4TY CONTROL PROCEDURES 

. 7-\ ' * . . 

Based on this analysis, a strategic or modular approach to the development of 
quality control procedures for the OSFA student aid programs was utilized. It 
adapted ttd various steps in the strategic approach to quality control) development. 
These steps can be broken down into two phases— design and implementation. The 
basic steps in the design phase were: 

+ 

•' Identify significant sources of error in the program (completed during 
- * * Stage One); 

m Conduct a functional analysis of the operating system^ncluding infor- 
mation' requirements, linkage structures, and breakdowirpoinjs. In other 

words, identify the places in the system where corrective action can be 
taken and monitor progi»ess; 

• Conceptualize a quality control system with modular components 
designed to detect and monitor error-prone functions. ^ 

... . ^ • > • $ , 

The steps in the implementation phase were: • 

• Select and prioritise modular subsystems for development. Also, 
identify time frame for specif ications, design, development, and installa- 
tion of each selected subsystem} ^ - ' .* 

• Proceed with systems development for selected subsystems (e.g., develop 
procedures and systems manuals, user manuals, system specifications, 
and software specifications, as necessary); 

t 

• Perform system tests on modular subsystems-as they are .completed. 

During the design phase, the focus was on a QC measurement system for the 
PelJ' Grant Program. As a result of a functional analysis and a comparison of Title 
IV Programs, suggesting many similarities between programs, all three major 
programs were included, in the basic design. As a result, the preliminary design was 
expanded based on a comparative analysis of the' Title IV programs to provide a 
basic framework for' the overall OSFA Quality Improvement Program. { 
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The implementation phase was characterized by technical assistance rather 
than detailed system design per se.^Yhis was undertaken in recognition that OSFA 
divisions must be responsible for quality improvement in their own areas of 
responsibility* The technical assistance was targeted on error-prone areas that 
heeded QC enhancements and were designed to provide an ongoing management 
report. - * ■ • • . 
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• CHAPTER 3 

DESCN^FOR THj£OSFA QUALITY IMPROVEMENT PROGRAM 

* • . • - ■ 

m During the past three years, OSFA has initiated an internal quality improvement 
program. This program has used the strategic approach to quality** improvement to 
- identify and correct error-prone activities in the current delivery system. This effort 
has focused om ...;..•.;.•*« 

• Identification of opportunities for quality improvement in the overall 
deiivery system; 

• Davefepment, of an overall strategy for\ improving the quality of the 
current delivery system? 

• Implementation of internal mechanisms for improving student aid delivery. 

This chapter describes the results of this effort and presents the long range plan 
for improving quality ift the student aid delivery system^ using the general approach 
discussed in Chapter 2. The chapter has four sections. First, the opportunities for 
/quality improvement a<e discussed. Second, the overall strategy for quality improve- 
ment is reviewed. Next/ the framework for the internal quality improvement program 
js reviewed. Finally, the steps in impiementation of the overall strategy are discussed. 

*' ' • ' > 

OPPORTUFfOT^fFQ^pUALnY IMPROVEMENT 

The development of the overall quality improvement strategy in OSFA was based 
on a detailed analysis of specific opportunities for quality improvement. A three-level 
framework was used to assess quality control needs of OSFA. Figure 3-1 presents an 
information system's perspective of OSFA based on this framework.* The three levels 
are: 

• The policy-level analysis examines the type of quality control information 
required by entities that interact with OSFA In setting policy for student 
aid. These actors include the remainder of the Department of Education 

1 Adapted* from Anthony. Planning and Control Systems? Ay^ramcwork for 
Analysis, (Boston, MA: Harvard Business School, 1965). See also S.C. Blumenthal. 
Management Information Systems (Engle wood Cliffs, N3.s Prentice Hall, 1969). 
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i t i? C °* Mana g«*ent and Budget (OMB), and Congressional 
staff. Interactions between OSFA and these entities usually occur as part 
of the annua] budget process or legislative reauthorization. P 

J * The ^anagemenVJevel analysis reviews the organization of OSFA and its 
current management procedures related to quality control. The analysis 

dSSSS^T 1 ?W 01 qualit t ****** mf ormationToct 

dures required for the effective management ol the Pell program. 

. • The delivery^evel ^aiysis considers the functional role of each orcaniza- 

™t1f I? ^ th ^ OSFA : ***** the control pS^ 

^SSS^L^Jff? to each ^ ity » the ^"nation resources ^twTthe 
entity, and the information needs of the entity. J 

: At the policy level, quality control concerns relate primarily to overall program 
delivery. Outside of OSFA, in ED and OMB, there is a Concern that payment error in 
the Pell program, for example, should ^-assessed on an annual basis. Additionally, 
analysts in the Office of Planning,, Budget, and Evaluation (OPBE) are interested in 
receiving timejyr^naiyiis 0 f policy options for corrective actons that require 
legislative or budget action. These same concerns are apparent within OSFA, along 
with an accurate^wartnesi that ^ritiail parts of the policy cycle ({banning, budgeting, ; 
andjegulation) and' program delivery process (application through reconciliation) must 
be performed on a timely basis. , N * 

At the management level, quality control 1^ th e lack of 

information on the quality of system performance. The functional organization within 
OSFA makes programmatic improvement more difficult, especially when activities in 
one Division require 4a*ormation and actions from -other Divisions. The lack of 
information about error-prone points canlimit the ability of managers to track critical 
activities and make improvements. The timeJy flow of information between functional 
units is often limited, in part due to the quality of the overall information system. 
Too often the information needed by another Division, regardless of its importance, 
takes second priority to performance of key activities within die Division. As a result, 
there is a need for an overall strategy that identifes critical error-prone points in the 
delivery system and provides a framework for ; implementing corrective actions, 
especially as they relate to the flow of management information within OSFA. 

Ar the delivery level, edsting quality control procedures are more 'abundant. 
Occasionally, Branches and Divisions have developed their own quality control 
processes, usually on an ad hoc basis. As a result, there is wide variant in the ' 
awareness of staff within OSFA about quality control issues. Some Branches have 
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deveipped fairly sophisticated approaches to quality control while other/ are st$U ^ ' 
plaguld by basic problems. Unfortunately, none of these procedures are tied together 
in an overall system with a well defined flow of information/ 

As part of this project, OSFA sponsored an exhaustive study of the eff ecf^Jjhe " m 
current delivery system for the Pell, GSL, and Campus-Based programs.^ It reveaHa . 
that the*gnost seriously negative effects elf the current system for the Federal" 
government are the unavailability of irif oenation and lack of fund control. 3 Most of 
the causes of these negative effects can be improved through marginal changes to the 
current' system. For example, most of the basic pre y ems with fund control for 
* student aid programs can be corrected b^improvements to Internal accounting^ 
procedures. For example, to the GSL program, where fund control is the most severe 
problem, changes in accounting procedures can ameliorate many of the deficiencies 
identified by the General Accounting Office.* Other marginal changes for GSt, suclv 
as increased use of state Quarterly reports in quality control checks for state claims 
and collections activities, could also reduce fund control problems. 

Quality control measurement can play an important role in the overall strategy 
for delivery system improvement. Using the strategic approach to QC system ^ 
development, it Is possible to put quality control checks into place for critic** \ 
activities' throughout the student aid, delivery system. Such development provides 
senior policymakers with the early warnings that they need about major system X 
problems, as well as provides program managers with the detailed information they 
Require to develop strategies for correcting delivery system deficiencies. For 
example, during the past year the GSL Branch) of the Division of Program Operations 
put into place a new system for ^measuring error in the manual interest payment V 
process. This enabled OSFA to find errors before they were uncovered by end-of-year % 
audits and, in turn, to make needed corrections. 



2 Advanced Technology Inc., and Westat, Inc. Assessment of Alternative Student Aid 
Delivery Systems? Assessment of the Current System, prepared for the Credit 
Management Board, EDTancT toe Division of Quality Assurance, OSFA, Dune 1983, pg. 
3^13. • ' " • 

3 Ibid, pg. 3-14. - L 

.^ The Guaranteed Student Loan Information System Needs a Thorough Redesign. U;| 
General Accounting Office, September 24, 1983. - r - 
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The importance of improving the quality of information about the financial 
management of Federal student aid programs is echoed in the findings of the 
President's Private Sector Survey on Controls- Task Force Report on Education .? The 
task force identified the need for improved management information systems and 
internal controls as a mechanism for reducing waste, fraud, abuse, and error.? The 
report considered the need for specific improvements In student aid programs and 
delivery, including improved debt collection. However, a major emphasis of the report 
was on the improvement of management information and fun# accountability, two 
closely related problems. • 

Short of an overall delivery system redesign, which takes substantial resources- 
and several years to implement, improved quality control holds' the most prjpmlse f or 
addressing these basic problems. Marginal improvements tan resolve many of the 
basic problems with the student aid delivery system. Quality Improvement efforts 
should be specifically targeted on improvements in fund control and availability of 
program information since these areas represent some of the most basic problems 
facing the Department. 

THE OVERALL STRATEGY 



OSFA has^developed an overall strategy for quality Improvement that can be 
implemented on an incremental basis and that targets resources on the most critical 
Problems. The overall design of the quality control program is illustrated In Figure 
3-2. The overall strategy has four major components: 

• Institutional Quality Control/ which encourages institutions to make 
delivery system improvements; 

* f~ 

• External Quality Control,, which provides quality assurance functions for 
institutional QC and provides ^overall measures of error; 

• * * 

• Internal Quality Control, which is designed to facilitate marginal improve- 
ments to the current system; V V 

• A Quality Control Management Information System (QCMI5) that uses 
inputs from the above cited sources to tailor reports for: 



5 



Submitted to the Subcommittee for Consider atioji at the meeting on 3une 13, 1983 



6 Ibid, pg. vii. 

3-5 38 



Institutional 

O Quality 
Control 



, ■ 1 



rr 



im- 



policy Level 
QC Reports 

Corrective Action Analysis 

Carly Warning Reports 

Summary Error Analysis 



External 
Quality 
Control 





Internal 




OSPA Quality 




Control 





Quality Control 
Management information 
System (QCMIS) 



I 



Management Items 
QC Reports 

• Identity Critical QC targets 

• Track Critical /Activities 

• Monitor System Performance 



1 



Delivery Level . 
QC Reports 

Improve Quality of Information 



• Improve Efficiency 

• Improve Productivity 



0 

ERIC 



39 



FIGURE 3-2 

DESIGN FdR Q5PA QUALITY CONTROL SYSTEM 
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OPY AVAILABLE 



' - . Policymakers requiring corrective action analysis, early warnings of 
system problems, and summary error reports * 

. Managers concerned about identifying specific QC targets, tracking 

V critical activities, and monitoring «veralJ system performance 

„ ■ ? ' ■ r ; 

- - Program personnel requiring basic routine information to improve the 
deliver y process and financial accountability.; , 

Each of these components of the overall design for the OSFA quality control 
program is discussed brief ly below. The interVial component is discussed in more detail 
in the next section. > i. 



Institutional Quality Control j « 

' • » . ' . • . t • : ■ 

It is the goal of OSFA to encourage the development, of systematic internal 
quality control, through measurement of discrepancies and errors in the delivery j 
system, in all postsecondary institutions participating in OSFA student aid programs* 1 
OSFA is taking two actions to facilitate this type of development. 

First, OSFA is considering regulatory changes that Woulg require postsecbndary , 
institutions to sample Campus*Based Recipient records to measure error. The 
objective of this requirement would be to encourage institutions to measure errors in 
their own delivery system and to develop corrective actions if they have»high error 
rates. Over the long term, OSFA will require institutions to measure and correct 
discrepancy of errors for all Title IV programs, not }ust for C^pus-Sased. 

Second, through the "target of opportunity* approach initiated during Phase U of 
this project, Advanced Technology developed a handbook outlining quality control 
procedures for the financial aid office. A draft of this document was reviewed by 
members of the student aid community. Currently, the handbookls^being revised to 
reflect OSFA and community concerns about long-term quality improvement at the 

» 

■institution level. This emphasis would place responsibility for quality control with the 
institution. 



External Quality Control 



Until institutions can implement a comprehensive institutional quality control 
plan, it is necessary for OSFA to develop and maintain a systematic process for 
measuring error in the entire delivery system. This is called external quality 




control. The external component of the OSFA Quality Improvement Program focuses 
on the three major program areas? Pell, GSL, and Campus-Based. The Pell Quality 
Control Study, conducted by Advanced Technology and Westat, Inc., was a compre- 
hensive study of error in the PeH program. A systematic framework for replicating, 
the Pell Study was jdeveloped, and currently is being implemented. The Campus-Based 
segment of external quality control can be implemented either independently or in 
.combination with the' Pell Quality Control-' studies* : ■'■'}:,* 

The^ measurement of error in the delivery of the GSL program' presents special 
problems that could not be. handled through modification of the Pell Quality Control 
4 studies. Since GSL involves a complex network of lenders and guarantee agencies as/ 
weira> institutions, applicants, and applicant families, it will be necessary to take a 
different emphasis for this study. It would, % at a minimum, include: 

... . \ ■ 

• Measurement of applicant error through a method similar to the Pell study? 

- * ' • • '* . • .. . / 

• Measurement of institutional error through methods similar to the Pell 
study? I 

' . / 

• Measurement of guarantee agency and lender error through an entirely new 
approach. .-^ • * • ' -4 ' "]' '' 

The goal of OSF A is to reduce its role in direct externa! measurement -.once an 
institutional quality control plan is operational. At that time, OSFA*s role will become 
one of quality assurance. Quality assurance for external measurement is th?prScess 
of monitoring the effectiveness of Institutional quality control programs and determin- 
ing institutional compliance with ED quality control regulations. 

■ * - ' • 

internal Quality Control 

The lon| range goal of the internal quality control program for OSFA is to 
develop QC measurement and reporting mechanisms for critical activities throughout 
the student aid delivery system. During Stage H, Part I of the Pell QC Study, th^ 
following activities were selected for technical assistance using the strategic frame- 
work: - \ - * / 

^ ■ " ■ . " 

— . « 

• OSFA Goals and Objectives System ^ 

• GSL Manual Interest Billing 
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• F1SAP Processes for Campus-Based Programs 

• External Quality Control System 

• Vendor/Processor Quality Control 

• mstitutional QC Guidelines. 

Subsequently, additional targets have been added. One such target initiated 
I during Phase Hi deals with the GSL reinsurance process. ' 



Quality Control Management Information System 

■i - . •.• v * . ' . ■ 

The major feature of the entire QC system is a quality control management 
information system (QCMIS) that receives inputs from ail of the above quality control 

^subsystems and can be used to generate reports for all levels of management. 

' ■# ■•• * .' i •*''•'•••» ' 

Conceptually, the QCM15 will include a framework for routine reporting from all other 

measurement subsystems, internal and external* as well as special analyses commis- 
sioned by the Assistant Secretary as part of the corrective action irameworkjfcwhicb is 
discussed 'in the next chapter. Most of these reports can be generated by the other 
modules, the QCMIS should consist of a defined set of information flow and reporting 
specifications for each of the modules. / > 

The framework for the routine reporting from other measurement modules to the 
QCMIS is presented in Appendix A. . This framework includes: 

. * 

• Summary reports from the external measurement module; 

• Summary reports from the vendor/processor quality control and goals and 
objectives modules; 

1 ■ . ' ■ \ 

• Sample report formats for the internal QC module. 

4 

The quality control management information system, when fully developed! will 
provide management reports for managers throughout OSPA. It is the objective of the 
Division of Quality Assurance to continue with the development of the QCMIS 
framework, as outlined in Appendix A. X 
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INTERNAL TARGETS FOR QUALITY IMPROVEMENT 

v * 

* * 

The purpose of the internal OSFA quality improvement program is to establish , 
mechanisms for "measuring critical error-prone points in the delivery system and 
improving the way tfiese activities are accomplished, if necessary. The section, 
presents ah analysis of critical activities in the student aid delivery system, the 
analysis recognizes the following features of the Federal role in the delivery system: 



There are more^ similarities than differences in the delivery systems for the 
major student aid programs, which suggests that an integrated approach to 
Qp is desirable? / 



I 



• Some critical activities are best addressed at a management level in OSFA 
since they cut across the entire delivery system; * 

• Other critical activities relate specifically to individual subsystems and 
programs. x_ ~ 

\ 1 

First, there are similarities across the delivery systems for student aid programs 
by . design. ' The Office of Student Financial Aid has a functional organizational 
structure that combines activities for many functions. Very often the same personnel 
work on all three' programs. The current delivery system for the Pell, GSL and 
Carnpus-Based program has six subsystems 




o 

ERIC 



Pre-application, which refers to the program planning and bSulget develop- 
ment, information dissemination and other activities that Usually take 
place prior to the start of an award year; 

Student applicat i on, which refers to the actual processing of student 
applications) 

0 ■ < * . • 

Student eligibility determination, which refers to the process of determin- 
ing categorical student eligibility and need for each program; 

Student benefit calculation, which refers to the process of calculating and 
packaging the awards 

- ' - * ■ _' 

Fund Disbursement, which refers to .the process of disbursing funds from 
the Federal government to students, institutions, guarantee agencies, and 
lenders; * * . ; 

Account Reconciliation, which refers to the process of reconciling all 
accounts — for students, institutions, lenders, states and the Federal 
government -- after the award year. For GSL, thiols by definition a long- 
term process since the Federal government subsidizes interest on loans 
after they are made. 
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For the purpose of internal quality control in OSFA, the three student systems — *■' 
application, eiigibilfty determination, and benefit calculation — can be treated a single-" 
subsystem. The Federal government has relatively little direct involvement in these 
subsystems, except for the Pell -program application process. Instead, carripuses and, 
to a lesser extent, lenders and guarantee agencies are the primary actors in these 
subsystems. 

"Second, there are some critical activities in the delivery system that cut across 
all subsystems and are most appropriately addressed at a management level in OSFA. 
Two of these critical activities were addressed during Stage U of. the Fell Grant 
Quality Control Study. One is vendor /processor quafty control and quality assurance* 
Private! contractors play * an important role in each subsystem. As a result* 
vendor/processor quality assurance cuts across subsystems. One of the technical 
assistance activities in Stage H, Part 1, was the development of guidelines for 
vendor /processor quality control for large processor contracts"* in OSFA and the 
development of a quality assurance manual for-prbject officers for these contracts. 

Another activity that clearly cuts across all subsystems is the Goals and 
Objectives System in OSFA* . This system is used to monitor all delivery system 
activities for each program. During Stage H, Part l ot the QC study, the Goals and 
Objectives System was also selected for technical assistance. In this area, the 
network approach to program martagement was pilot tested, and found applicable to 
the OSFA program management. This approach has not yet been Implemented. 

Third, there are numerous activities in the delivery system that are candidates 
for quality control and quality improvement that are most appropriately addressed on 
an activity-by-activity basis within subsystems* The purpose of this section is to 
identify the critical activities, using a sdithd. overall framework. Critical activities are 
delivery system procedures that are particularly error/ prone. The framework used to 
•identify critical activities: 

Defines the subsystem; 
Identifies the goals of the subsystem! 
Identifies the quality control objectives, for the subsystem? 
Identifies the relevant quality control measures; 

Identifies critical activities for quality control that correspond to these 
goals, objectives, and measures. 
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In the remainder of the section, the framework is applied to the four key 
subsystems in the student aid delivery* These are: 

• Pre-appUcation Subsystem? « 

• Student Application, Eligibility Peterminatkm, Benefit Calculation Sub- 
systems? * 

• Funds Disbursement Subsystem? 

• Account Reconciliation Subsystem. 



Pre- Application Subsystem 

r • 

. <• v » j 

The pre-application subsystem is the process for out year prdgram planning. It 

includes all activities related -to development of forms, budgets, and application 

information. Quality control in* the pre-application subsystem is extremely important 

because this subsystem represents the design stage in the (delivery system. Deepens 

made at this stage will affect program quality at nearly all subsequent stages. In 

other words, the quality of*the student application, eligibility determination, ahd 

benefit calculation subsystems, funds disbursement subsystem, and account 

re^nciliation subsystem are, at least in part, dependent en the quality of the policy 

decisions formulated in the pre-application subsystem. Therefore, effective quality 

control in the design stage can prevent many problems at subsequent stages. One 

mechanism for ensuring quality control in the pre-appllcatlon subsystem is en 

independent program design review council that couid provide management oversight 

on an ongoing basis. 

The quality control goal of this subsystem' is to conduct all pre-application 
activities in an efficient, timely, and responsive manner. Quality control .objectives 
and/neasures are divided into those related to formulation of policy and those related 
to implementation of policy, this distinction is made to highlight the need for 
preventing potential error through quality control of the policy formulation^process. 
The critical Federal activities at the delivery fcjyeljiii the pre-application subsystem 
are identified in- Figure 3-3. The reasons for inciug|ng these targets are considered 
below. ^ 

Of the targets already selected by OSFA, only the two management level QC 
'targets .— vendor /processor quality assurance and the goals and objectives system — 
explicitly address the pre-application subsystem. Vendor processor quality control 
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PRE^ APPLICATION \ 

The process that Includes forecasting and developing budgets, developing and 
promulgating federal regulations, developing necessary forms, disseminating 
program information to participants, training participants, contracting and 
planning for services, determining institutional program eligibility, establish- 
ing payment systems, and planning for program specific procedures. 

Goals ■ . " ' 

* • \ '. .* - • .. . 

Conduct all pre-applicatlon activities in an efficient, ; timely, end responsive- 
manner. 

Quality Control Objectives; £** 

• ' ' ' i : " ' X P ■ " ■ • ' 

Policy Formtf lation • $ ^ 

Review major policy changes > 
Estimate impact'.of major policy changes 
Management oversight on mafor policy chaises 

Policy Implementation , 

TimelyJo'eveiopment.of regulations 
Timely promulgation "of regulations 
Timely development and printing of forms . 
Adeo^te availability of information *: v ' 

Timely mforpation dissemination r 
Timely and accurate responses to telephone and mail inquiries 
Timely determination of initial Pell authorization levels 
Accurate determination of initial Pell authorization levels 
Timely determination of Campus-Based allocations 
Accurate determination of Campus-Based allocations 

Quality Control Measures: 



Policy Formulation - 

Absence or presence of review on major policy changes 

Percentage increase or decrease in program error 

Absence or presence of management sigivoff on major policy changes 

Policy Implementation - ' 

Number of months from iinalization of regulations to beginning of* 
program year 

Number of months from prbmuigation of regulations to beginning of 
program year ' -f 



FIGURE 3-3 

CRITICAL ACTIVITIES IN THE 
PRE^APPLICATION SUBSYSTEM ^ 
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Number of months from development and printing of forms to beginning 
of program year V° , " 

Number of months from forms development and printing to beginning of 
program year 

Number of months from information disseminatiqn to beginning of 

program year * . 
Comparison between participant information needs and availability of 

information that is disseminated * 
Accuracy of answering participant questions by monitoring telephone 

calls and sampling mail responses - • 

Number of days to complete initial authorization for Pell 
Difference between initial authorization and final allocation in Pell 
Number of days to complete authorization for Campus-Based 
Percent of Campus-Based allocations that are appealed 
Difference between tentative allocation and final allocation in Campus- 
.Based. •» 



Activities fljr Qua lity Controls 

• ED development and promulgation of new regulations for each program 
as required 

• ED development of forms including application forms and instructions, 
authorization letters* requests for payments and reimbursements, SARs, 
progress reports, validation rosters, FlSAP, loan assignment forms, and 
teacher cancellation lawns ^ 

• ED dissemination of program information through dear colleague letters, 
participant training, responses to participant inquiries, and development 
of handbooks and manuals * 

• ED initial authorization of institutional funds activity for Pell by DPO 
through 'tfie PeU Disbursement System 

• ED tentative through final allocation of institutional funds for Campus- 
Based programs through FISAP processing in DPO. 



, FIGURE 3-3 (Cont.) 

CRITICAL ACTIVITIES IN THE 
FRE-APPLICATION SUBSYSTEM 
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relates to pre-application since the processor contracts; such as Peii, must be changed 
each year to incorporate new requirements, which * is a process that should -be 
monitored trough vendor processor quality assurance. The Goals and Objective 
System explicitly ackijowledges the planning (or pre-application) cycle, as well as 
delivery and wrap-up cyde of any program year. ; 

Regulation development is important from a design perspective. The major 
design concepts are put into action in regulations, which highlights the importance of 
design reviews. A design review can be critical in many instances, ft is possible the 

"criteria-Cfcn be established internal to OSFA, to determine when a management review 
should be; done. The developmenj and promulgation of new regulations is often viewe©* . 
as an important area of concern by trie student aid community. The major problem, 
from the j community's perspective, .with this activity is the timing of new regulations . 
relative to the time and costs of implementing them in the field. If there is ample 

^ead time, campuses are more likely to be able to adjust to new regulations as part of 
a routine planning process. However, when new regulations come out just prior to or 
oaring the award year, the probability of error is increased due to the fact that)' 
adjustments must -be made at the last minute, if they can be made at ail. A-QQ 
analysis of this activity should be done from the perspective of the impact on the 
entire delivery system. Key Measurement points coulo K ^Ljde ntified that would 
provide OSFA managers and policy makers Insight into possibleproblems before* they 
happen. 

The development and revision of forms is also a critical factor In filling out the 
right information at the right time. The Goals and Objectives System partially address 
the forms development process, since it is identified in this system. However, the 
impact of delays in these key forms has not been systematically analyzed.' ft is 
possible to develop a mechanism for monitoring the timeliness and availability of 
important forms. 

Another critical pre-appiication activity the dissemination of program lnfor- ' 
mation through various sources. There was substantial criticism of OSFA during the 
recent public hearings that important information is simply not available when needed 
and that when it is available it is often contradictory* For example, state guarantors 
who operate in different states have observed that different regional offices interpret 
GSL regulations differently. During site visits, campus financial "lid administrators 
observed that they often have to make numerous calls to OSFA to get answers to 
simple questions about the Federal end of the delivery system. This problem cuts 
across Divisions in OSFA and, therefore, shouloSe addressed at the management level. 
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The initial authorization of funds for the Pell program is another critical activity 
that can be tracked through the quality control system* This activity sets §n initial 
allocation which can cause majjoi^ problems for the institution if it is too low and cause 
an excess of float if it is too Arge. A coherent and systematic approach to monitoring 
this activity fc^uld be developed within the Pell Grant branch, DPO. ?, ■ 

Student Application, Eligibility Determination, and Benefit Calculation Subsystems 

The Federal government is not directly involved inmost of the activities 
0, included in these subsystems- Therefore they were combined for this analysis." These 
subsystems represent the actual interface between the student and the delivery 
system, which takes place at several points. The Federal government is only directly \ 
involved in^this subsystem for the Pell program. The quality control goal of this 
subsystem is the timely processing of> applications and the accurate determination of 
eligibility and benefits. V * 

The critical activities ^or this program are identified in* Figure 3-4. These are 
the processing of applications, determination of eligibility and benefits for students 
applying to ADS institutions, an* validation procedures used /to verify selected 
application data items. All of t he l ratfo yities have already at least been partially 
addressed by the quality control study. *\ 

The processing of applications for Pell was treated explicitly in the report of 
vendor /processor quality control Procedures^or inwoving quality control were 
actually built into the new Pell contract. , - { r*^ 1 

ED determination of eligibility for students attending^ADS schools is the only 
delivery system activity that involves a direct interface between O^A and students. 
Numerous quality control procedures, including sampling and error measurement, can 
be implemented for this activity, just as they can be implemented at the campus le&el. 
The campus level quality control guidelines that are. currently being developed for 
OSF A can be adapted to this function. The validation procedures' used by schools for 
RD5 and ED for ADS is also, addressed in the institutional quality control guidelines. 

.-•4 * 

t 

Fund Disbursement Subsystem 
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Fund disbursement is'the process of disbursing program funds from the Federal 
government to state agencies, lenders, institutions, or students, and. from institutions 
to students. The quality control goal of the funds disbursement subsystem is to 
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STUDENT "APPLICATION, ELIGIBILITY DETERMINATION, AND BENEFIT 
CALCULATION 

The process by which a student applies for financial aid either through a 
processor or institution, the processor or institution reviews the application 
for compliance with eligibility requirements, and award amounts of maximum 
loan amounts are determined 

Goal: 

Timely processing of student applications and* accurate determination of 
eligibility and benefits. 

• *■'•.- 
• ■• - ■ 

Quality Control Objectives: 

Timely initial Pell processing 
Timely Pell corrections processing 
, Reliable Pell processing * 
Accurate Pell categorical eligibility determination . ~V 

Accurate Pell benefit calculation 
Accurate Pell validation \ 

Quality Control Measures: 

Number of days between application receipt and mailing of SAR 
Number of days between corrections receipt and mailing of corrected SAR 
Percent of edit identified errors by Peil processor that are actual errors 
Number of transactions per applicant 

Percent of applicants ruled eligible who actually meet eligibility criteria 
Percent of bene! it calculations computed accur^tefy 
Percent e£ei|gibiliity determination errors remaining after validation 

Activities for Quality Control: 



• Processing of student applications, calculation of SAI, generation of SAR 
by Pell processor * 

• ED determination of elig&ility and benefits for students applying to ADS 
institutions* 

• Validation procedures to verify SARs used by schools in RDS and by ED 
in ADS. 

i 

* Activity already at least partially addressed. 

FIGURE %4 

CRITICAL ACTIVITIES IN THE STUDENT 
APPLICATION, ELIGIBILITY DETERMINATION AND BENEFIT 
CALCULATION SUBSYSTEMS 
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transfer the right amount of money to thejcorrect recipient at the proper time. 
Critical activities are identified in Figure 3-5. ^ 

Quality tontrol procedures have already pttn developed for two activities in the 
funds disbursement subsystem* interest and sjpedal allowance payments in GSL and 
disbursement of Campus-Based funds through the F1SAP. Quality control procedures 
for interest > and special, allowance payments monitored, the accuracy of disbursements 
from ED to lenders and guarantee agencies byi examining the efficiency of the receipt 
control process for 799 forms, the. completeness of submitted 799 forms, the accuracy 
of calculations on the 799 form, whemerVdupUcate payments were made to lenders or 
guarantee agencies, and the accuracy of Treasury vouchers authorizing payment. 
Processing timeliness was also monitored by ensuring that the date on the certification 
letter or Treasury memorandum for a voucher preceded the penalty date for a 799 
form. This process has helped identify problem areas in the interest payment and 
special allowance procedures in need; of corrective action and allows subsequent 
monitoring of the impact of the corrective actions. 

Quality control work for PBAP concerned developing corrective action 
strategies for error-prone functions. Analyses were conducteo^jpf the most common 
errors committed in FlSAP reporting and problems with FISAp- procedures and forms. ' 

ED disbursement of funds' for the Pell, program to RDS schools is a critical . 
delivery system activity. It takes place as an integral part of the Pell Disbursement 
System. During the past year, the Pell Grant Branch of DPO has made several 
improvements in the accuracy of the process. The purpose of the quality control 
modeTwould- be to monitor the timeliness and a^uracy of this process and to report, on 
the financial implication of this process. 

The ED disbursement of funds to schools in the Campus-Based program happens 
as a drawdown process. This activity is most appropriately addressed as an accounting 
issue since it currently has no mechanism, except audits, for determining if funds are 
being drawn down according to Federal regulations. The ED accounting system does 
not currently track these requests on a program basis. The routine tracking of this 
activity could result in significant cost saving due to the tightening of the float for 
institutions that draw down funds sooner than allowed by Federal guidelines. • 

ED disbursement of funds to ADS students is primarily an issue f,or reasons of 
timing. The Stage One Pell Quality Control study revealed that the actual disburse- 
ments for the ADS students are relatively accurate. However, the timing of 
disbursement can be problematic for ADS students and institutions. A quality control 
mechanism for this activity could monitor both timing and accuracy. 
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FUND DISBURSEMENT \ 

The payment of program funds frojn the federal government to state agencies, 
lenders, institutions, or students, and from institutions to students. 

V " * ■ 

I * 

Goals 

Transfer the right amount of money to the correct recipient at the proper 
time. ■ • . ' / 

Quality Control Objectives: 



Timely disbursements 
Accurate disbursements 
Minimize float 

Atcurate in-year program cost estimates 

i 

Quality Control Measures 



, — v Number of days between scheduled disbursement date and actual disbursement 
' V date \ 

Difference between actual disbursement and "correct" disbursement \ %f 

Percentage of disbursements in error 

Amount and, imping of disbursement to institution compared to amount and 

timing of dispursemont to student 
Difference befween afctual program costs and estimated costs for program 

year ■ x 

Activities for Quality Control; . - . 

• ED disbursement of funds for the Pell program to RDS institutions and 
> ' RDS students through the Peii Disbursement System 

• ED disbursement of funds for the Campus-feased program to institutions 
through the FISAP process* ... . \ j 

' I - 

• ED disbursement of funds for the Pell pro^m to students f in ADS 

institutions through the Pell Disbursement Processor 

• ED payment of interest to lenders participating in the CSL program 
while student borrower is enrolled, in grace period, or is in deferment 
period and ED payment of special allowance to lenders to subsidize 
Guaranteed Student Loans* 



* Activities for which control has already been at least partially addressed. 

' ' V 
FIGURE 3-5 

CRITICAL ACTIVITIES IN THE FUND 
DISBURSEMENT SUBSYSTEM 
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ED payment of administrative cost allowances to guarantee agencies 
participating in the GSL program to compensate them for. servicing costs 

Refunds in Pell program funds from institutions to ED for students who 
graduate early, withdraw* or drop below half-time enrollment during the 
time covered by the grant. 



( 





FIGURE 3-5 (Cont.) 

CRITICAL ACnVTttES IN THE FUND 
DISBURSEMENT SUBSYSTEM 
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ED payment of administrative cost allowances to guarantee agencies in the GSL 
program is another critical activity. ED currently has no mechanism for monitoring 
the accuracy and timeliness of this disbursement. It is possible that data reported to 
ED on quarterly reports could be better used for this purpose. 

The collection of refunds in the Pell program is a critical activity that is 
difficult for ED to Monitor. When there are changes in student enrollment status, this 
usually means that a refurfa is due to the Federal government, since most changes are 
for reduction in course loadNSurections to the SAR are usually not reported until the 
end of the year in the SVR. Analysis of this activity could focus on QC enhancements 
to the Pell Disbursement System. 

Account Reconciliation Subsystem • « 

The account reconciliation subsystem includes all processes required to reconcile 
program accounts for a given award year and collect loans. The QC goal in account 
reconciliation process is to eliminate incorrect payment through record reviews and 
audits and reduce loan default rates. The account reconciliation process provides a 
back end quality control for the entire system. 

The critical activities in the Account Reconciliation Subsystem are identified in 
Figure 3-6. None of the quality control technical asistance provided during Stage H 
directly addressed the aa^m recohcilliation subsystem. It is, therefore, a subsystem 
with several opportunities for quality improvement for internal quality control. 

ADS account reconciliation for student accounts is one of the critical activities 
in the account reconciliation subsystem. ~\ The objectives of a quality control 
enhancement for this activity would be to develop a routine measure of the difference 
between the actual disbursement and what should have been disbursed. 

Another critical activity is reconciliation of institutional accounts for Pell 
through i!he Pell Disbursement system. The problem with this activity is delays in the' 
reconciliation process, possibly due to 4 the absolute standard used for reconciliation. A 
QC study of this activity could focus on improved tolerance levels and standards for 
the reconciliation process. 

A closely related activity is reconciliation of institutional accounts for 'Campus- 
Bank programs through the FISAP process. The verification of FISAP can be 
approached through detailed analysis of consistency on the FISAP form. This has been 
done on an ad hoc basis in the^oast by the Campus-Based Branch, DPO. A routine 
sampling procedure could be used. 

* * 
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ACCOUNT RECONCILIATION 



The process of reviewing records and determining that the amount of program 
funds disbursed to each student, institution, lender, or guarantee agency is 
correct, and, the process of student loan repayment. 



Goal: 



Eliminate incorrect payments through record reviews and audits and reduce 
loan default rates* * \ 



Quality Control Objectives; 



Accurate recapture of ADS student overpayments in Pell 
Timely recapture of ADS student overpayments in Pell 
Accurate recapture of institutional overpayments in Pell and unused funds to ' 

Campus-Based ..' " ^ 

Timely recapture of institutional overpayments in Pell and unused funds in - 
Campus-Based 

Accurate information on enrollment status for students with FISL loans 
Accurate payments by lenders and guarantee agencies to ED for collections on 
defaulted loans 

, Timely payment by lenders and guarantee agencies to> ED for collections 
defaulted loans % * 

Accurate recapture of overpayments ort claims for defaulted loans 
Timely recapture of overpayments on claims "for defaulted loans 
Accurate information on borrowers teaching in low-income schools or in 

military or "Head Start" service 
Timely collections on NDSL leans assigned to ED 
Correct collections on loans assigned to ED 
Accurate institutional audit and review procedures 
^Accurate guarantee agency audit and review procedures 
Accurate lender audits and reviews 
Accurate call reports and lender manifests in FISL 
Timely caU reports and lender manifests in FISL 

Accurate quarterly reports, administrative cost allowance letters, and tape 
dump for GSL ' 

Timely quarterly reports, administrative cost allowance letters, and tape dump 
for GSL . ■ • 

Quality Control Measures: 

Difference between ADS student overpayment and ED collections 
Number of months between end of award year and closing student account 
Difference between institutional overpayments and ED collections 
Number of months between end of award year and closing institution account 
Elapsed time between change in enrollment status and receipt of information . 
by ED for FISL borrowers 

Difference between actual lender or agency collections on defaulted loan and 
m amount reported to ED 

Percent of collections on defaulted loans reported to ED in error 
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ACCOUNT RECONCILIATION SUBSYSTEM 
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Number of days from lender or agency collection to transfer to ED 
Percent of ofaim overpayments recaptured 

Number of months from identification of claim overpayment to recapture- of 

overpayment ^ 
Percent of borrowers with Cancelled loans for which employment status data 

are accurate 

Percent of loans assigned to ED Upon which collections are made 
Percent of institution^r auditors and reviewers following required procedures 
Percent of guarantee agency auditors and reviewers following required 
procedures 

Presence of required data elements on call reports and lender manifests 
**' Percent of lenders meeting ED reporting requirements for call reports and 
lender manifests 

Number of days between due date of call reports and lender manifests and 

submission date • 
Completion of required data elements on quarterly reports, administrative 

cost allowance letters, and tape dump 
Percent of guarantee agencies meeting ED reporting requirements for 

quarterly reports, administrative cost allowance letters, and tape dump 
Number of days between due date ,of quarterly reports, administrative cost 

allowance letters, and tape .dump and submission date 

Activities for Quality Control: * 

Student account reconciliation for students enrolled in ADS institutions 

Institutional account reconciliation through verification of student 
validation roster in Pell and through FlSAP in Cam^fs-Based 

Enrollment .status reporting to determine if loan is eligible for interest 
subsidies for FISL and GSL * ^ * 

Recapture of overpayments to lenders and state agencies on defaulted 
loans and capture of collections made by lenders and guarantee agencies , 
on defaulted Idan§ * 

■9 

NDSL cancellation when a borrower is teaching in a low-income school 
or is in military or "Head Start" service and assignment of NDSLs in - 
default for two years to the federal government for collection 

Institutional audits by independent auditor and program review by ED 

Guarantee agency audits by independent auditor and program review by 
ED 

FISL lender audits by independent auditor and program review by ED 

FISL lender reporting through call reports and lender manifests 

Guarantee agency reporting through quarterly reports, administrative 
cost allowance letters, and tape dump 
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Enrollment status reporting to determine loan eligibility for interest subsidies is 
critical fpr FISL and GSL. For FISL, the timing of requests for enrollment verification 
has a built in problem for the Federal government; the bi-annual reporting for 
enrollment leaves too little time between' verifications for students who drop out right 
after the report is made. For GSL, enrollment reporting is an extremely complicated 
process of concern to lenders, GAs and institutions. A number of quality improve* 
ments are possible in this process. The default rate for GSL and FISL could be reduced 
through such an effort. 

For the claims and- collections process in GSL and FISL the government lacks a 
systematic approach to check the reasonableness of claims. Therefore, the Federal 
government currently lacks the capacity to correct on overpayments. This is an area 

* - * 

where an enhancement study would result in substantial savings. 

NDSL cancellation and assignment of NOSL to the Federal government is 
another activity that has not had quality. control checks in the past and where 
improvements are possible. A QC enhancement in this area could include a systematic 
sampling of cases to determine overall error rates. 

Institution audits by independent auditors and program reviews by ED is another 
activity in which quality improvement could result in savings. While a school 
monitoring system for the DCPR review process is in the design stage, it has not been 
implemented and ED presently lacks the capacity to do analysis of sources of error. 
.This information could be used for all aspects of OSFA management, from regulation 
development through account closeout. 

Guarantee agency audits and program reviews is another activity where duality 
improvement could possibly lead to financial savings. f ED currently lacks the systems 
auditors needed for thorough program reviews for GAs. A QC study in the area could 
more precisely identify the review needs for GAs. 

The ED review of lenders, suffers from a lack of 'a sound information system that 
can be used for analysis purposes. While lending institutions are generally highly 
regulated, there is now a real possibility that when lenders do submit a bill with errors 
it will go undetected by the Federal government. A GSL QC study may be necessary 
to define and measure lender error to provide a benchmark for the audit and review 
processes. 4 

Guarantee agency reporting is. another activity whece quality improvement is 
needed. GAs report though quarterly reports, administrative letters and state tape . 
dumps. XMe- quarterly reports could be better Utilized by OSFA for QC checks on 
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various aspects of the GSL reconciliation process. The state tape dum^ also offers 
possibilities it) this area, but has not been consistantly reported to ED. The quality of 
data on all three reports, has not been routinely checked by OSFA. The entire OSFA- 
state interface is an area where quality improvement is needed and a X}C enhance* 
ment study is possible. ' '** 

THE NEXT STEPS 

OSFA has made substantial progress during the past three years in the design and 
implementation of an overall quality improvement program. The emphasis of the 
program during the first two years has been on the measurement component. During 
Stage One of the Pell Quality Control Study the emphasis was on error measurement 
for the Pell program. During Stage H the emphasis shifted to internal quality 
improvement within OSFA. During Stage Three, OSFA has maintained a dual emphasis 
on internal quality improvement and external measurement of error in Pell. 

During Stage m of the Quality Control study, a strategic approach to internal 
quality improvement was ^developed, tested, and used to select high error prone 
targets. This chapter has reviewed. those targets, identified overall strategies for 
quality improvement and identified new targets for. quality improvement. Possible 
next stejSjs in this internal quality improvement process are to; 




Select additional targets for technical assistance 

'V- '« # 

• Develop procedures for internal QC development by operating units in 
OSFA. • « ' 

• Develop the QCMIS reporting system. 

* 

First, it is now an opportune time for selection of additional targets for quality 
improvement. This chapter has presented a set of critical delivery system activities 
that can be selected for technical assistance during Stage Three of Quality Control 
Study. Work so far during Stage III, Part II, of the QC study, has included GSL 
reinsurance .and refinement of the institutional QC guidelines. 

Second, the OSl-A should also consider development of guidelines for overall 
quality improvement in OSFA* A framework for this activity is considered in the next 
chapter. * ^ . 
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Finally, OSFA should also consider development of a <^CMIS tor routine reporting 
on all quality control and quality improvement activities. The framework for this is 
presented in Appendix A. 



\ 
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CHAPTER 
IMPLEMENTATION PLAN 

\ v 

This feport has described the results of Stage Two, Part I of the PellTOC 
Study, a design and technical assistance effort, and has focused on the measurement 
component of the OSFA Quality Improvement Program. In order to achieve the 
overall objective of this project, to design and implement an ongoing quality 
improvement program, OSFA must continue this systematic quality improvement 
effort. In order to facilitate the process, this concluding chapter considers: 

• The overall Quality Improvement Program design; 

• The implementation of the Qual^y Improvement Program? 
^ • The development of a corrective action component; 

>• Priorities for action. N \ % 

- ■ ' : ■ . ; ■ * 

OSFA- QUALITY IMPROVEMENT PROGRAM 



OSFA has made a concerted effort over the past three years to identify and 
implement quality control measurement and to organize corrective action analysis. 
The Pell Grant Quality Control Project has propose a wide range of corrective 
actions. These proposals have not adequately involved OSFA personnel, nor has a 
formal structure for corrective action been proposed previously as a result of the 
current project.. This section considers the basic elements of a generalized quality 
improvement program for OSFA. 

Quality improvement should be a goal of all Divisions in ^SFA, not just DQA. 
Each Division should have an internal quality control plan. As part of this plan, 
records and documentation should be maintained and summary reports on 
performance sent to DQA. DQA's role in this* process should merely be a monitoring 
or oversight one. Therefore, each division is responsible for quality control. Their 
roles can be distinguished from the responsibility of DQA whose responsibility is 
quality assurance. 
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The quality improvement program has two basic components; the technical 
QC system with an emphasii on measurement, and the analytic component with an 
emphasis on corrective*ections. The major emphasis is in the designtask on the 
technical measurement component which attempts to identify error-prone points in 
the financial aid program. Having identified- these points, tjre objective pi the 
corrective action component ^is to introduce program reforms which can increase 
overall progcam quality. The steps in the technical measurement and corrective 
action components, as well as the interaction between these components, is 
illustrated in Figured. Quality control has been ^defined as a process^o>- 
identifying, correcting, and preventing error or a tendency toward error in a system. 
Therefore, as shown in figure 4-1, a formal ongoing quality control effort must 
combine both a technical (or measurement) component and an analytic (or 
corrective action) Component. ♦ # 

The technical component of the quality control process is already well into the 
design stage. Figurfc 4-1 shows that the technical component must include a 
capacity to: , ' 

• Define samples? 

'.':»'' ' ' \ 

• Define measures; 

• Establish standards; 

• Develop measurement mechanisms; 

• Impiemenfmeasurement mechanisms. 



* The analytic or corrective action exponent of the overall quality control 
procedure is essential to close the loop and repeat the quality improvement cycle on 
an ongoing basis. The critical elements of the corrective action component, as 
shown in Figure 4-1, are: * 

» ( 

% ' ' i ' 

♦ ' 1 ' 

• Comparison of actual performance with standards; 

.... . . '. « . . .. i 

• Management selection of corrective action options; 

• Implement corrective actions; I 

• ,-» 

• • Repeat the cycl^^ •*' * . 



.'V •' 
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FIGURE 4-1 



STEPS IN TIB5 DENpOffffiOT* OF m 0SF7V 
QUALITY IMPROVEMENT PROGRAM 



* The development of a formal corrective process in OSFA would require signi- 
ficant changes in OSFA management. Some of the assumptions that were 
considered in the development of this framework were: 

' • The ongoing quality improvement program, especially the corrective 
action component, should be integrated into the overall management 
system in OSFA; T, 

• The corrective action framework must interface with other management 
systems and procedures irt OSFA, including the Performance Monitoring 
. System (PMSfc ; 7 



It is important that the corrective action component provide OSFA 
personnel with opportunities to initiate and receive recognition for 
v corrective actions initiative, increased productivity, and error reduction; 

• A management commitment to quality in OSFA is essential to the ' 
* implementation of the overall QC system? 

• The implementation of the corrective action component of the Quality 
Improvement Plan should"be an integral part of the ongoing QC system. 

When proposing that a formal organizational mechanism be established for the 
ongoing quality improvement program, it is necessary to consider the organisational 
intent in which the system will be implemented. In OSFA, features of the current 
system should be recognized in the overall design. These includes 

• The formal management structure in OSFA and current initiatives to 
improve management; 

• The placement of the new system in the organizational hierarchy. 

The Office of Student Financial Assistance is organized into functional 
divisions with responsibility for different aspects of the delivery of all three 
programs. Divisions are further subdivided into branches. In some divisions, such 
the Division of Program Operations and the Division- of Policy and Program 
Development, there are separate branches for each major program (Pell, GSL, and 
Campus-Based). In others, branches are divided by function. For example, the 
Division of Certificatiorvand Program Review has five branches, each, with responsi- 
bility for different functions. In the larger divisions, there are sections .and units 
with further refined sets ffKMflponsibilities. Within this hierarchical structure, 
OSFA is in the process of implementing several significant management enhance- 
ments. ' .A 
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One of the major management enhancements that is currently being imple- 
icnted in OSFA is the Goals and Objectives System. This system identifies goals, 
oHjeciCives,. activities tasks, subtasks, and steps for the delivery of each major 
student aid program. This system has the potential of strengthening the manage* 
ment of individual programs asj a complement to the functional' management system 
that is currently in operation. It identifies units ant individuals responsible for 
completing individual steps. Currently, OlSFA is exploring * networking approach to 
the Goals and Objectives System which will improve its program management 
capability. 

Another significant management enhancement being implemented in OSFA is 
the Performance Monitoring System (PMS). PMS will provide branches and sections 
in OSFA with a formal mechanism for: * 

• Identifying performance measures for individuals, units, sections, and 
branches? 

• Reporting on routine performance of work activities; 

• Establishing goals for improving performance within units; 
Monitoring performance of indivi^ls, Units, and sections; ^ 

• Evaluating performance of individual employees based on established 
criteria; 

• Recognizing and rewarding exceptional performance. 

# * 

This system involves employees in establishing criteria and setting perform-' 
ance goals. It is being implemented in branches where the work activities are of an 
ongoing repetitive nature, such as forms preparation or review* 

Both systems will provide OSFA (with an improved management capability. 
However, in spite of these innovations, the management structure in OSFA remains 
a hierarchical structure with a top-tibwn information flow a^out policy and 
procedures, and a bottom -up flow of information about work performance. Conse- * 
quentiy, the link between policy formulation and actual work activities in OSFA is 
undermined by the absence of information about the types of, actions that can be 
taken to improve error-prone areas and functions in the delivery system. The 
quality control process, with its technical and analytic components, is intended, to 
give OSFA a formal mechanism for strengthening this link. Both operating unhs and 
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OSFA management will participate in this process in distinct ways. Operating units 
will develop and implement their own quality control plans, the impact of these 
plans on quality performance will be monitored periodically by management through 
a quality "assurance function* Formal coordination of the quality control and quality 
assurance functions could be assigned to \ specially created body such as the Quality 
Council described later in this section. 

The quality improvement program is Intended to provide OSFA with technical 
and analytic support for the overall OSFA quality improvement process. The overall 
quality improvement process must be well integrated into the management structure 
of OSFA, as well as provide a mechanism for improving the linkage between policy 
formulations and organizational activities. An illustration suggesting the placement 
and the role of the corrective action process is presented in Figure 4-2. This 
placement and role is explained more fully in the following discussion. The basic 
design of the quality improvement process with ' its technical and analytic 
components, should enhance the Voles of OSFA managers in instituting corrective 
processes in their units. " % " " ' 

IMPLEMENTATION OF QUALITY IMPROVEMENT PROGRAM 

The design of the OSFA quality improvement program must consider the 
formal organizational mechanisms required to put the framework into action. Four 
formal mechanisms are critical to implement the overall quality control process and 
the corrective action framework. These ares 

s. '. - 

• Assign Responsibility— an individual or group must coordinate the correc- 
tive action process?' 
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Develop Procedures for Initiating Corrective Actions— procedures must 
be developed to give OSFA personnel the opportunity to gain recognition 
for identifying corrective action options; \ 

Develop Procedures for Implementing Corrective Action—a formal 
mechanism for implementing new procedures related to the corrective 
actions component of the quality control process must be developed; 

Develop a Reporting System Interface QCMI5— the corrective actions 
framework must be integrated with the technical aspects of the quality 
control system. 
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' Assign Responsibility 



There are two optional approaches for formalizing the quality improvement 
program in OSFA. The. first would simply require appointing a senior official to be 
responsible for the implementation of the overall program in OSFA. This official 
would designate staff responsibilities for implementing the process and for working 
with divisions and branches. . 

Alternately, the Deputy Assistant Secretary for Student Financial Assistance, 
in conjunction with Division Directors and Branch Chiefs, could appoint an OSFA 
Quality Council. The membership of the council, in combination, might include: 

• Representatives from each of the OSFA Divisions; . 

• Individuals ^with responsibility for the areas identified earlier as targets 
for increasing progfam qualify and reducing error in each Division or 
Branch; 

• At least one representative from each program, GSL, Pell, and Campus- 
Based. '..*•' 

«- • ' 

The Quality Council should be responsible for the overall quality control 
process, both the corrective- action component and the technical component. The 
core component of the Council should consist of five to six senior managers since 
larger groups are more difficult to convene and manage. The core Council will be 
augmented by special subgroups assigned on a case-by-case basis to resolve 
particular problems. These subgroups will analyze the source of the problem, report 
back to the Council on perspectie corrective actions, and) subsequently be dissolved 
at the completion of their work. The DQA should provide staff assistance to the 
Quality Council. The purposes of this council should include additional: ' 

• Responsibility for the OSFA quality assurance function; 

• Approval of Branch and Division quality control plans (with particular 
emphasis on cross-divisional implications); 

• Responsibility for implementing the corrective action process and devel- 
oping OSFA policies and procedures for implementing and initiating 
corrective actions. Y 
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Since OSFA managers are joined on the Council by ad hoc subgroups of Division 
staff to meet particular; needs, the Quality Council represents a combined manage- 
ment-staff initiative to maximize program quality. Management and staff working 
together to meet common goals can have numerous residual benefits to the 
efficiency of OSFA programs. 



Corrective Action Procedures 

• , • * 

9 

' ■ ■ .; . ( . • 

Another important mechanism for the ovefcail quality control process will be 
the development of a procedure for establishing" corrective actions. This procedure 
should be established by the Quality Council or senior official in charge of 
corrective actions. * V. 

When establishing procedures for initiating corrective actions, the Quality 
Council' or responsible individual should recognize the different types of corrective 
^actions that can be taken. Specifically, the council should distinguish between:- 



ERIC 



'\ • Type 1— Working leveJ corrective actions that can be implemented at the 
! spot where the worked is performed. Usually, the Section Chief or 

Branch Chief can approve this type of corrective action. The Quality 
Council or responsible individual should be concerned primarily with 
reporting of these actions to the QCMIS as a formal mechanism for 
monitoring marginal changes? * >v 

• Type 2— Corrective^ action analysis should be required for marginal 
changes that/effect more than one division or have implications for the 
overall delivery system for one of the -programs. The Quality Council or 
responsible individual may reserve the option to approve the selected 
option, in such cases, the corrective action analysis should consider 

effects of the current procedures on key participants, 

\ ■ . ' 

effects of options on key participants, 



selected corrective action; 

Type 3— Pr6gram level corrective actions should be analyzed by the 
Quality Council or responsible individual. t Again, the effects of the 
current system should be analyzed along with the differential effects of 
the marginal change options considered. Recommendations should be 
submitted to the Deputy Assistant Secretary for approval; 

Type 4— Policy level corrective actions usually require actions outside of 
OSFA. Some marginal changes, such as forms redesign or simplification, 
require policy decisions at the level of the Secretary and above. In these 
cases, the Quality Council or responsible individual should submit 
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recommendations^ to the Deputy Assistant Secretary. The Quality 
Council or reponsible individual should consider the types of additional 
procedures that vfouid be necessary fbr Type 4 corrective actions. 

i ■ . . 

The Quality Council wili attempt to continually improve program performance and 
will introduce corrective actions for this purpose. The Council will not merely 
recommend necessary changes when program quality deteriorates to the point that 
it is not meeting specified standards. In other words, continued increase in 
performance will be the goal of the Council, rather than mere maintenance of the 
status quo. 

.., ' f, ' ?' 

Procedures for Implementing Corrective Actions 

Whenever a corrective action has been approved, new-forking procedures are* 
necessary. It is possible that these are never documented. Very often, formal 
written procedures Will be necessary. For example, when the GSL Branch, DPO, 
undertook corrective actions in the area of manual interest billing, the branch 
instituted new procedures. At the very least, the Quality Council or responsible 
individual should establish an overall procedure for implementing corrective actions. 
This should includes 

• Description of the corrective action; 

• Description of activities* tasks and steps affected by the change (perhaps 
related to the Goals and Objectives System); 

• Documentation of written procedures that are to be changed; 

• Notation of new procedures that should be developed. 

QCMIS Reporting 

The corrective action system should feed directly into % the quality control 
management information system. The corrective action reports to QCMIS should 
consist of: .... 

• Summary reports from Divisions and Branches; 

* 

+ Corrective action analysis reports; 

• Corrective action implementation reports. 



These reports will vary in frequency and purpose. Summary reports should be 
designed to provide periodic reporting on progress. Along with the implementation 
of the corfecjive action process, the reports could provide the basis of the 
corrective action reporting system* The type of corrective action analysis reports 
used would depend on the types 'of corrective actions being implemented according 
to the above framework. The corrective action implementation reports would 
provide a mechanism for reporting cm the effects, or savings, of implementing each 
corrective action. Formal report formats could be developed for each type of 
report. - 

v 

CORRECTIVE ACTION COMPONENT OF OSFA 
QUAUTY IMPROVEMENT PROGRAM 

The development of the corrective action component of the ongoing quality 

improvement prograsri would represent a significant departure from past practice in 

OSFA. It would require building a commitment across OSFA to quality 

* -i •' ' 
improvement. As ^discussed earlier, litis would- require a formal designation of a 

senior OSFA official or other person to be responsible for implementation of the 

quality improvement program, especially the corrective action component. The 

implenj^ntation of the corrective action component would require implementing a 

corrective action process. ' ^\. 

The senior official appointed by the Deputy Assistant Secretary, or Quality 

Council, should initiate an annual quality improvement program in OSFA. This 

requires a significant investment in analysis. John L. Kidwell * observes: 

A quality improvement program is an investment; manpower wUl be 
spent if results are expected. In order to put everything into the proper 
perspective, a fact-finding activity is a prerequisite to this investment. 

One good way ofr involving the whole organization - in this phase is 'to 

give key managers^ specific responsibilities in the fact finding. The 
recommended approach is through an appointment of an ad hoc team. 

V * 

Kidwell recommends that the ad hoc team have the same membership require- 
ments as the Quality Council recommended in the previous section. The basic steps 
proposed by Kidwell for the annual quality improvement analysis/plan are: 



*3ohn L. Kidwell, A Profit Plan for Quality , Waterford, Ct.: The John L. Kidwell 
Company, 1975. , 
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- Determine quality policy and currentcompliance with quality policy; 
Determine or estimate quality costs? 
Identify dominant quality problems; 

Determine compliance. to the operating gfiits quality system; 
Identify the existing defect prevention system; 
Collate and analyze findings; 
Develop recommendations for unit management.* 

*'-.• • - • • 

These steps are used here as a basis for an action plan for implementing the 
OSFA corrective action system. The action plan is outlined below. 

Determine Quality Policy and Compliance 

Determination of the OSFA quality-policy could be one of the most critical 
tasks of the Quality Council. Kidwell defines quality as "that degree of excellence 
of a product or service, that provides for full customer satisfaction over the. 
expected life," with timely; availability at a cost to the customer that he can afford, 
and at a profit to the procedures".^ Clearly, this definition would have to be 
modified to fit a definition of quality lor student aid delivery systems. It is possible ^ 
that the QC objectives' in each subsystem, identified in Chapter 3, could serve as a 
basis for developing the OSRA quality statement. Once the definition is developed, 
the Quality Council initiates the action plan. The first step in the action plan is for 
the Quality Council to requesf Division Directors to: 

4 ' ■ 

* i 

• Review OSFA quality definition and program requirements; 




Determine the applicability of the definition to their organization; 

Determine whether or not improvements in the operation are required to 
adopt the quality policy; 

Determine the extent to which current documentation an8 procedures 
are adequate. 
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Divisions and Branches may identify significant improvements that shouid be 
made. For larger Branches, the Branch Chief may need to involve section chiefs and 
unit chiefs in the "program. The information generated from this process 'will 
provide a starting point for developing the corrective action system. 

Estimate Quality Costs j 

.•I 

• ** . , ■ 

Costs of Quality (COQ) is a concept Kidweil recommends for highlighting and" 

displaying the "cost of unquality." According to this view, the concept of quality 

cost management is a simple one— "once you know these costs, you can take steps to 

reduce those costs that offend you".* In order to achieve this type of incentive 

structure, it is necessary for the units to reports ^ 

m Costs of quality program (either a QC module or system enhancements 
developed internally); «*-.'.*' 

fc . • Costs of losses caused by nonconformance to standards. (This , will 
require fuller specification of loss or gains*) ••» 

This type of analyst can help Division and Branch Chiefs to fdentify areas 
where corrective actions can be implemented. For example, during the past year, 
both the Pell Branch and GSL Branch of DPO have implemented marginal changes 
that could result in substantial savings. The establishment of this type of reporting 
system would provide a formal mechanism for giving recognition for such enhance- 
ment. 

The specific critical activities identified in Chapter 3 can provide a basis for 
this step. The focus of this preliminary analysis shouid be on identifying the amount 
of error in the system. - \ 



identify Dominant Problems ' - 

' \ • 

The basic question here is, "what needs to be fixed?" Quality costs teU^ 
managers the areas where improvements are needed. The Pareto approach to 
corrective action analysis can be applied. This approach recognizes that, SO percent 
of the problems are caused by 20 percent of the cases; therefore, the Division and 

^Ibid, p. 36. . * 
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Branch managers should focus [on their, most error-prone areas. They should be 
asked to identify their own dominant quality problems, their seriousness and 
magnitude. 1 They should be asked to separate the "vital few*' problems from the 
•trivial many." Corrective actions should be directed toward important problem 
areas. - ' , 

This list of critical activities also provides a basis for this step. OSFA 
managers, or perhpas the Quality Council* could go through this list to identify 
priorities for continued action. 

Determine Compliance to Unit Quality System 

■....! : - ■ ' 

This will be a two-step process. Since most units now have an overall quality 
improvement plan, it will be necessary to first develop a quality plan for the unit— 
this should evolve out of the review in the prior step. 

Generally, organizations have two quality systems: The one they think they 
have, and the ones they actually have. 5 In order to determine the actual quality 
system, it will be necessary to ask OSFA managers: 

, » * * 

• If the bask functions are being performed; 

• i |lf these are established standards for these activities; 
^ • \lf these standards are actually adhered to; 

• If the new standards are needed. 

\Kidwell recommends that the Quality Council establish a "Quality Audit 
Guideline." Once these guidelines are established, several quality audit teams 
shcLld be formed to the selected areas. The audit teams should have the 
responsibility in the area being audited. 

1 In OSFA, audit teams should work on those activities that are considered the 
highest priority. Technical assistance may be desired for some high priority targets. 
The emphasis should be on . establishing and reiining reasonable tolerance and 
standards foe these critical activities. 
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Determine the Defect Prevention System 

Defect prevention, especially marginajNsprrective actions, should be the 
responsibility of "Operating units. The corrective action procedures outlined in the 
previous section should help establish this (principal in OSFA. The Quality Council 
should consider whether the operating units have internal corrective action systems. 
The basic question that should be asked of air managers is: What are the things- you 
do, every day, in managing your workers, to prevent their making mistakes? The 
response will indicate the current defect prevention system in the unit. 

The process implemented by the GSL Branch, DPO for the manual interest 
billing and reinsurance activity should serve as a model for this type of corrective 
action. * In both cases, the Branch Chief took responsibility for identifying and 
correcting basic problems.- Each of the critical activities could be managed 
similarly by Branch Chiefs and other managers on a project basis. 

'. ■ r . • 

Collate and Analyze Findings 

A significant amount of information will be generated from the prior steps. A 
critical task is to put these results together Into a meaningful report! The report 
should focus on: -.. \ * 



\ 



• Cause and effect relationship; 



1 \ 



- \ • The effects of the current system; \ ^ 

• The marginal changes tha\ can be made to improve performance; 

\ 

• Assessment of the likely effects of possible marginal change. 

~Z Most of the seriously negative effects of the current student aid- delivery 

f " system can be improved through marginal improvements to the current system. The 
critical activities could all be the subject of -mis type of analysis. 

... * 

Install Ongoing Corrective Action System 

Once the Quality Council has been through this cycle, the basic parameters of 

the ongoing corrective action system can be defined and implemented. In fact, this 

■ A . 

* 
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\ ■ \i do, every day. In managing your workers, to p 

response will indicate the current defect prevention 



\ 



Determine the Defect Prevention System 

- * 

Defect prevention, especially marginal corrective actions, should be the 
responsibility of operating units. The corrective action procedures outlined in the 
previous section should help establish this principal in OSFA. The Quality Council 
. should consider whether the operating units have internal corrective action systems. 

\ \ . The basic question that should be asked of all managers U: What are the things you 

t their making mistakes? The 
system in the unit. 

The process implemented by the GSL/ferandh, DPO for the manual interest 
billing and reinsurance activity should serve as a model for this type of corrective 
action. In both cases, the Branch Chief took responsibility for identifying and 
correcting basic problems. Each of the critical activities could be managed 
similarly by Branch Chiefs and other managers on a project basis. 

Collate and Analyze Findings 

. A significant amount of Information will be generated from the prior steps. A 
critical task is to put thase results together into a meaningful report. The report 
should focus on: \ 

■ • * Cause and effect relationship; 

• The effectfr^f the current system; 

• TheinS^girfai changes that can be made to improve performance? 

• Assessment of the likely effects of possible marginal change. 

Most of th* seriously negative effects of the current student aid delivery 
system can be improved through marginal improvements to the current system. The 
crj/ical activities could all beNhe subject of this type of analysis. 

'Instail Ongoing Corrective Action System • : f 
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Once the Quality^Gouoci! has been through this cycle, the basic parameters o! 
the ongoing corrective action systfem^an be defined and implemented* In fact, this 

t 
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process can become an- annual corrective action process that can lead to ongoing 
refinement and improvement of the student aid delivery system. The implementa- 
tion of corrective actions in each of the critical activities Would not. lead to 
.improved overall performance of the delivery system* 

PRIORITIES FOR ACTION 

This report has summarized * two-year quality system design and technical; 
assistance task that has focused on the development of an ongoing Quality Improve- 
ment program for the Office of Student Financial Assistance. In order to assure the 

« 

continued implementation of the quality improvement program, the following, prior- 
ities for continuedaction have been identified: ' 

• Assign responsibility for quality program; 

• Design and development of quality control management information 
system? 

• * Implement corrective action component of quality improvement 



program; 

Continue to develop selected targets of opportunity. 



Responsibility for Quality Improvement Program 

It has been proposed in this chapter that the Deputy Assistant {secretary for 
Student Financial Assistance designate a senior official responsible for the OSFA 
quality improvement program. Alternately, a Quality Council can be appointed to 
oversee the implementation of the quality improvement program. 

Quality Control Management Information System 

As indicated in Chapter 3, the Quality Control Management Information 
System {QC MIS) is thecritical implementation link of the measurement component ' 

"orthe quality improvement program. The QCMIS shareJ emphasis: 

C 

• Refinement of reporting mechanisms from other OSFA quality control 
systems and subsystems, according to the framework that has already 
i been established; 
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• Systematic analysis of policy and management issues inherent in the 
results: of the ongoing measurement system; 

• Possible inclusion of the corrective action component/ ° 
Corrective Action Component 

: • . P 

: The corrective action component of the overall quality improvement should 
involve all organisational units in OSFA in a systematic qua^i^y improvement 
program. It is possible that data analysed as part of the Delivery System 
Assessment Task, Part II of Stage H, .could be used to analyze and prioritize 
corrective action options for each "delivery system activity. A blueprint for this 
type of involvement was discussed earlier. , The reporting mechanism developed 
from this process could be incorporated into the QCMIS. 

Targets of Opportunity 

■ , ■ * „ ■ f 

In addition to continuing development of the reporting mechanisms across the 
OSFA QC system, several of the targets of opportunity already considered are in 
need of continued technical assistance. Chapter. 3 identifies a list of possible 
targets. These targets are summarized in Figure #-3. This chapter proposes a 
strategy for developing a systematic framework for developing corrective actions 
for each target. The goal of OSFA should be to establish a corrective actiorf agenda 
using this framework. 
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Vendor/Processor Quality Control and Quality Assurance Guidelines 



Goals and Objectives System 



Institutional Quality Control Guidelines 



FISAP Processing for Campus-Based Programs 



\ 



External Quality Control System 
GSL Manual Interest Billing 



GSL Reinsurance Claims and Collections 



Program Regulations Development and Revision 
Forms Development and Revision 
Program Information Dissemination 
Pell Program Funds Authorization 

M * 

Pell Program Validation 



Pell Program Funds Disbursement 



Administrative Cost Allowances in GSL 

Pell Program Refunds Collections 

Student Account Reconciliation in ADS Institutions 

Institutional Account Reconciliation in Pell Program 

Institutional Account Reconciliation in Campus-Based Program 

Loan Eligibility Determination for Interest Subsidies in GSL 



Reasonableness Check on Insurance and Reinsurance Claims 



NDSL Cancellation and Assignment to ED 
Institutional Audits and Program Reviews 
Guarantee Agency Audits and Program Reviews 



Lender Audits and Program Reviews , 
Guarantee Agency Reporting ' ; 

NOTES: ' 

Targets in BOLD indicate activities already at least partially implemented. 



Targets with 1 BOXES I around them indicate activities that should be . the highest 
priority. . * 



» FIGURE 
* 

SUMMARY OF QUALITY CONTROL TARGETS 
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This appendix provides a framework for developing the OSFA Quality Control 
Management Information System (QCM1S). The concept behind the QCMIS 
framework is simple but effective. -Based on • the Pareto principle, which 
hypothesizes that 20 percent of the^ cases cause eighty percent of the problems, a 

% strategic approach to QC development in GSFA was developed. The strategic 
approach assumes that the most error -prone areas should be selected for QCMIS 
development, and puts the SO/20 principle into action by targeting corrective 
actions on the 20 percent of the cases with the biggest problems. This strategy 

Worked effectively during Stage II of the quality control study. 

< When fully* developed- the QCMIS should provide a defined set of inputs, 
processes and outputs -for eacK component, or module, in the OSFA QC systerft. This 
appendix 'focuses exclusively on the outputs of the systerh. The QCMIS, as explained 
in Chapter 3, would consist of routine management reports on QC rtieasurement of 
critical activities in OSFA* Jn addition to reviewing these report formats; this 
overviey/ provides a framework for the Deputy Assistant Secretary for Student 
Financial Aid, or a Quality* Improvement Council, to select new targets for QC 
.development.,, , ^ m ■ 1 

The overview is divided into three major sections. The first section considers 
the report formats for ttfe external quality control system. The second reviews the 
report formats dnd options t»r the two' management level quality control 
subsystems. Finally, the third section proposes fit framework for identifying the 
quality*, control measurement that can be developed for the OSFA student aid 
delivery system. This section includes examples of report formats 'and sample 
.applications of the fr'qttewofk* An objective of Stage III of the Pell Grant Quality 
Control S?udy will be To develop fully the specification for the OSFA Q.CMI5. The 
purpose of this appendix is to -identify report formats for^he QCMIS. It does riot 
specify the sy^tetn. Tfcfe final section does, htiwever, provide a framework that can 
be used fo spec'if y the QCMIS- for critical delivery system activities. v . 

external Measurement- system 

The long-range goal of Xhe OSFA Quality Improvement Program is to Havtf 

♦ ■ * ... 

ongoing externalfQC systems iot all three^pajor program components:^ Pell-, GSL* 



and Campus-Based. Currently only one of these has tbeen developed—the Pell 
external QC measurement system.- Stage l] ^jj^the Pell QC study actually provided a 
measucement of the error levels lor Pell. As.part ol Stage II of the Pell QC study, 
guidelines were developed for ortgoing Pelt QC measurement systems, During^Stage 
III of the Pell QC study, this study is being replicated. It is possible that the Pell 
QC study could be extended to include the Campus-Based programs. The GSL 



programs would require a separate QC study since it. would probably include lenders 
as well as applicants and postseconda 
three components are described below. 



as well as applicants and postsecondary institutions. The report formats for the 



For the Pell Gr^nt Program, the key summary report for the QCM1S would 
xgive.Ihe^summary e5tiijnation< ofvirvstitutlon and student error. The format for, this 
report,- as presented jn th'e executive summary of Stage 1 of tfie.Pell <^nt QC 
study, Figure A-T, includes surmtiary estimates for the net and gross number dollar 
error, the percent af individuals^ or .institution^ with errors, and the rtftetfh error 
amount. The table also provides an indication, of the amount of overaward and 
underaward. The Pell QC studies, of cpurse, proviff^moFe detailed- results including 
specialized analyses of policy issues* but these basic summary reports can provide ^ 
the basis for QCMIS reporting. Additicfhal sumfnary'itables would^provide a basis for 
QCMIS reports which include: . * / * ' 

• Breakdowns for the size Of disbursement errors (sefe Figure A-2 for an 
illustration); 

r . 

'.At » . 

• Impact of selected incorrect Application itetos on grant disbursements - 
(illustrated -in Figure A-3); * * v x * ' 

- • Summary of institutional errOr incidence (illustrated in Figure A-4). 

. ■ . * ' . . .• 

For Campus-Based programs, the table structure would be similar, althougjlfo^ 
/He definitions of -error, both, for applicants and institutions, would ^^yV;.; •'" 
considerably. As part of Stage III -of the Pell Grant QC study, Advanced Technology*' ' , " 
is refining a list of error for institutional amd ^student errors lor the Campus-Based 
programs. When developed, these can be further 'refined into summary* report * 
formats for the QCMIS. r * . • - " * 



DOLLARS 



'ALL LRROR Z 
% Of RCCiriLNTS 



MEAN' 



DOLLARS 



HL TERROR 
X OF RLCIPILN1S 



Mf AM 3 



Institution Error 1 $210 H 

Student Lrror 1 • $39S M 

Total of Student A 

Institution Errors $605 H 

Total Caso Error $5£0 M 



37X 

4n 

71% 4 
7 IX 4 



$ 09 

M67 

$256 
$241 



$ 10 H 
$301 M 

$311 M 
$J12 M 



•37J 

4 n • 

7IX 1 
71X 4 



<s> $ « 

$120 

$132 
$132 



OVERAWAROINO ERROR 
DOLLARS X OF RECIPIENTS MEAN 5 



DOLLARS 



UNDERAWARDING ERROR 
X Or RECIPIENTS 



nr. an 5 



Institution Error' . 

Student Error* 

Student and 

Institution Errors 

- Total Casfr Ewor 



$110 H 
$340 M 

'$450 M 
.$440 M » 



20X 
36X ' 



49X 1 
49X 1 



$237 
$412 

$396 
$370 



$100 M 
-$ 47 H 

-$147 H 
-$120 H 



17X 
9X 

22X^ 
22X 4 



$243 
$237 

-$203 
-$249 



, M%s!n9 affidavits or statements of educational purpose and financial aid transcripts are not 
Included as Institutional error. Any cases with error greater than two dollars are Included. 

^Amount of error associated with all typfes of total Inst I tpHonal error plus all type* of student 
error per recipients totalled Independently. v 

^Hean for^all recipients. * . ' 



^Undupllcatcd count of Institution and/or student error. 
^Mfran for cases with error* . „ * 



• -7 



Source: Advanced Technology Inc. , and Westat, Inc. Quality In the DaSlC Grant Delivery Sy stem, 
Cxccutj^e Summa ry , p. 5.. *— — .■ 
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FIGURE A- 1 
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NOT INCLUDING AEP/FAT ERR0fr- r 198O-Olj . ; gjrgj ggpy flyA|JJ\g[£ 



\ 



AWARD ERROR 



ALL. STUDENT L 
INSTITUTION 
' ERROR 



PERCENTAGE OF CASES 

STUDENT £ 
INSTITUTION ERROR 
NOT INCLUDING 
-AEP/FAT ERROR 



( 

STUDENT ERROR 
NOT INCLUDING 
AEP/FAT ERROR 



$551 and less 

$251 to - $550 

$151 to - $250 

' $51 to - $150 

$3 to - $50 

$2 to - $2 . 

, $3 to $50 

$51 to $150 

$151 to $2^0 

$251 to* $550 

More than .$550 

Award Errors in 
Excess of $150 



Awa rd 
Excess 



liBbr 



rs in 
250 



2.3% 
4.9% 
3.4% 
5.4% 
4.2% 

26.3} 
* 8.1% 
- 8.6% 
7.5% 
12.2% 
17.1% 

47.4% 

36.5% 



2.4 
5.6 
3.7 
5.8 
4.4 
. 28.6 

8. a 

8.9 
7.9 
11.9 
12.1 

43.5 

0 

■ 32.0 



0.8 
2.0 

2. 



2.0. , 
53.7. 

5.0 

6.3 

6.1 " 
'-9.5 ' 
10.8 . 

30.5 

23.1 • 



Source: Advanced Technology, Inc., and Westat, Inc. Quality in the Be.sic 
# 6ran4Del iverv System, Executive- Summary , p. Zl 5 ] 
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' * DISBURSEMENT;* ERROR BY RANGES 



/ 



^ APPLICATION ITEM 


ERROR (NET 
" IN MILLIONS)} 


RESULTING 
INCREASE IN AWARDS 
PER STUDENT *NET)2 


Adiusted Gross Income 




" $53 




St udent /Sdou se 1979 Income 


43 


18 




Nontaxable Income (Other Than 
.Social Securfly) 


41 

* • * 


17 




* > 

Household Size 




15 




St udent /Spouse Assets 1979 


26 


11 




• 

Number in Postseconda ry 
Education Institutions - , 


13 

i 


, • 5 




— « * — i 1 1 

♦ r 

1 For policy purposes, the data 


from our sample are 


extrapolated to 





program-wide error levels. Note that there is substantial overlap of error 
amounts, so column total is larger than actual total student e«y» . Data 
.$re rounded to the nearest million* * 

2 Dafef aKe rounded to the nearest dollar* 



Source: Advanced Technology, Inc., arid W^stat, Inc. Quality' in the Ba$ic 
Grant Deli very System, Executive Summary , p. 8. 
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FIGURE A- 3 

1HPACT;*0F /SE^ EITED INCORRECT APPLICATION ITEMS 
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.'UN GRAN1 DISBURSEMENT ERROR 



1 *v 



-J 









/ 

•MEAN 'ABSOLUTE 


• 


ESTlMAtED 


PERCENT 


ERROR FOR 




RECIPIENTS 


OF ALL 


RECIPIENTS 




WITH FRPOR 


urn d t r UT C 


h J I H tRRDR 


fotal Institution Error 


991,000 • 


• 

« 


$364 


Instituti on'Error 


• 




- 


> 

without AEP/FAT E^ror 


873,000 


37% 


.$239 


Components^ 








AEP/FAT Erro'r 


181,000 


7.7; 


$933 


BA end Citizenship Error 


4,000 


• t m 




Program*El igi bil ity Error 


31,300 


£ 1.3% 




Cost of Attendance^ 


3*54,000 


15.05 


$177 










Enrollment Status Error^ 


430,000 • 


18. 2% 


$219 


Calculation Error? 


. 368,000 


15.6; 

t 


$ 79 



^Component figures are computed independently for eaj&h, type of error. 
The sum therefore exceeds the total of all error, bfcause error has been 
counted more than once in all cases where more than one type df- error 
occurs. ♦ . - 

^Estimated breakdown of "institutional err6r fiompohejats "ustng Spring 1981 
data. . Final component 'Wgures will be deriyed f romTlnstltutlpnal v .» f 
reconciliation . rosters as part of Sta^e Two of thi i project. - 

Source; Advanced Technology, lnc\, and Westat, 'Inc. Quality in thejtesic 
* Grant Delivery Svstem, Executive Su mmary ; p. : 

. 'T - ^ \ 1 : " — rr: — — _ , * • \ 



FIGURE A-4 



For the GSL program these basic report formats would have to be revised to 
include lenders and guarantee agencies as well ^is students. The definition of errors 
will a^so 9ary for the GSL program, 

MANAGEMENT LEVEL REPORTS 

The design OSFA Quality Improvement program design includes two basic QC 
measurement modules that operate at the managemegt level in OSFA. Both have 
elements that cut across programs and. subsystems in the delivery systems. These 
are a vendor processor qualjty control system and a goals and objectives quality 
control system. An overview of the types of reports that could be developed for the 
QCMIS* follows. 

Vendor /ProcessQr QC 

*. " . 
The vendor/processor quality control subsystem is actually comprised of a 
series of contracts for' the processing function for OSFA programs. The internal 
reporting forms developed for reporting on vendor/prbcessor contracts are contained 
In Figure AO. They include ^wo types o4 perfor^nce data: 



# Data on a series of key processing measures; 

• Data on exceptions (measures of performance that are operating outside 

of established tolerance levels). * 

s 

The form is designed for middle-level 'managers; therefore, only critical 
indicators of contractor performance are repprted sp that managers .do not have- to 
wa4e through excessive information^ firjdthe data they are ir^jpsted in. 

The section on key measure^seleCts a number o"f key prqpessipg activities (in 
Pell, for example, these nriight be applications processing, corrections prQcessin^, 
corresponderlfee serviqe, and fc^phQne service), and for each " one, reports 
information on: , * * . ♦ ^" * . 



• Processing volume; 

• Processing costs; 



v m & 

• 1* 

f • • _ 



• Error r^tes. \* j A .. . 
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j^Wtor's N; 



nme 



y'5 



•porting Period: l : rom 

To 



KIlY MUASIIKIl 



OSPA INTERNAL REPORTING SUMMARY PORM 
SECTION 1 - KEY MEASURES 
KEY ACTIVITY H KEY ACTIVITY 02 \ KEY ACTIVITY #3 



KEY ACTIVITY 0 'I 



Voluinc ln£oMnatjon 

'" • L 

oh 9 

Time (Average Processing/ 
Response Time) 

r^ost In formation - „ 

l\ascline Estimate 

* * 

Actual Cost 

[in or Rale 
Standard 

Actual 

9il ' C JL IS cy Mc a 5 vi res 



This Last Year 
Period Period to Date 
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Tills Last Year*' 
Period Period to Date 




This Last Year This 
Period Period to Date Period 



Last Year 
Period to Date 



§ tanda^jj 



This /Period 



Last Period 



Year to Date 



FIGURE A-5 



90 BEST .COPY Ava._«iBLE vendor?processor qc table shells 





/ 



t. PRODUCTION CONTROL: 



uSFA it^TEkiwL OR'm-»0 SImmMAIs • .OR., 



PROCESSING FUNCTION 



SECTION 2 - EXCEPTIONS 
/ f 

MEASURE OUTSIDE TOLERANCE 



STANDARD 



ACTUAL 



A. Information Receipt 



D. Data Entry 



C. Data Edit 
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BEST COPY AVAILABLE 



FIGURE A-5 (Cont.) . 
VENDOR/ PROCESSOR QG TABLE SHELLS 
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v)SF/i iHTEki'otL Kt.» OK'iihG SV^.t.MAU . . Oil... 
SUCTION 2 - EXCEPTIONS (CONTINUED) 



PROCESSING FUNCTION 



MEASURE OUTSIDE TOLERANCE 



STANDARD 



I. PRODUCTION CONTROL (CONTINUED): 
D. Compute 



E. Document Production and Mailing 

■■ - . - • / 



F. Cor recti orts~ 



v ■ 
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FIGURE A-5 (Cont.) 
VENOOR/PrtQf ESSOR QC TABLE SHELLS 
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BESfCOPY AVAILABLE 



PROCESSING FUNCTION 
' 2. FISCAL CONTI 



ViFA ft*#ERi<M%L Ru*i vlRT»itv» SUntmAR. « OKI. 
^ SECTION 2 - EXCEPTIONS (CONTINUED) 

MEASURE OUTSIDE TOLERANCE 



Bfesf COPY AVAILABLE 



StANDA^D 



ACTUAL 



1 



3. SOFTWARE QUALITY ASSURANCE 
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F I CURE A~5'«{Cont.) 
VENDOR/PROCESSOR QC TABLE SHELLS 



97 




OSFA fctTEBNAL REPORTING SUiwmAU i rORfn 
SECTION 1 - EXCEPTIONS (CONTINUED) 



BP" 



PROCESSING FUNCTION 



MEASURE OUTSIDE TOLERANCE 



STANDARD 



^PRODUCTIVITY CONTROL 

Km 

Manual Procedure A 



Manual Procedure B 



% 

V 



Manual Procedure C 



Manual Procedure D« 
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OSFA iNTEHrmL Rn-OAiimi SbmnJAR ■ ■'OR... 
SUCTION 2*- EXCEPTIONS (CONTINUED* 



PROCESSING FUNCTION 



MEASURJ!r"6uT5IDE TOLERANCE 



5. REPORTING PROCESS 



\ \ 



.V 



6. CORRECTIVE ACTION PROCESS 



» 



BEST COPY AVAILABLE 
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FIGURE A-5 (Cont.) 
VENDOR/PROCESSOR QC TABLE SHELLS / 



In addition, other key measures are presented and compared to predetermined 
standards to measure processing efficiency. (In Peli, other key measures may 
include the number of applications in the system more than 20 days, the number of 
corrections in the system more than 10 days, the percent of applications with more 
than olSe transaction, and the percent of edit identified errors that are real errors.) 
For each measure, data are reported for this processing period, the prior processing 
period, and the year-to-dkte. This use of trend data allows the identification of 
potential processing problems before they become actual problems and the prompt 
^initiation of correctiv^^actions. 

The section on exceptions is' divided into the various processing steps and 
substeps.. These steps and sufcsteps are: 



' Production control: 

'•■ " information receipt, 
data entry, 
data edit, 
compute, 

document production and mailing, 
corrections. J 

Fiscal control? 

c 

Software quality assurance? / 
Productivity control; r 
Reporting, process; • 

« 

Corrective action process. 



t * 



Since there art so many potential quality control measures in a 
vendor /processor contract,.' this section of the report is actually a table shell on 
which the monitor lists only measures that are operating outside of tolerance 
-ranges. Depending upon contractor performance, the number of exceptions may run 
from zero to several dozen. This' table shell approach provides a much more 
readable and useable format for managers than pre-printing ail possible performance 
"measures. ■ * 
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OSFA Goals and Objectives System 



The basic reports for the current OSFA goals and objectives system consists of 
Gahtt charts for all activities and tasks required to deliver student aid. As a part of 
the Stage II Pell Grant QC study, Advanced Technology did a QC enhancement study' 
of this system. This enhancement study focused on testing the network concept for 
the delivery of studenj aid programs. It was concluded that the network approach 
cdu|d generate a variety of summary management reports. These include: 

• A cost control Gantt chart fdr each phase of the process or function that 
displays the schedule status of activities to be accomplished within the 
Baseline Cost; Estimafe (BCE) funds available; 

> A cost ,status report that lists fixed and variable costs with associated 
, x activities? 

• A resource analysis report which describes, m dollars, the status of funds 
(budgeted or actual) within each cost center of the BCE? , 

\ • . a : 

• A cost-versus-time plot of budget-versus-obligated funds for each cost 
center of the BCE? 

• A critical path of the activities network jhat routinely reports planned 
and actual achievement of project activities. 

The OSFA Goals and Objectives system has the capacity to meet OSFA 
decision support needs, especially if the network approach is used. OSFA's decision 
support needs are illustrated in Figure A-6. Ail of these needs can be met by an 
automated networking system, such as the PRIME/VISION system Used fof.the QC 
enhancement study. 

«r ' * 

At a policy-level, the Goals and Objectives system using the network approach 
can provide summary reports on the status of the delivery system, status reports on 
programs and subsystems, simulations of decision plans, and resource tracking "and 
allocation reports. These reports can be tailored to meet the routine reporting 
needs of senior administrators or 1 to provide early warning of system breakdown. 
Such decision support systems can. also meet the detailed reporting needs of middle 
managers and operations managers," 
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^'sumwa£y reports • | 

Status reports on programs and / 
subsystems . \ 

simulations. of decision '., / 
options < what if?) * ' / 

Tracking and allocation 




pROGRE*£ TRACKING AND SCHEDULING 
fNTERDEpENDENCY VISIBILITY 

Identification of future task/ 
AcfTrwiei - 
Task/activity resource 

REQUIREMENTS f ; * 



Day to day activities/ 
tasks/steps 

Precise responsibility 
identification. 
Identification of 
required interfaces y 
( inputs and outputs), 
Progress evaluation 
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OSFA DECISION SUPPORT NEEDS? 
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DELIVERY LEVEL QC MIS FRAMEWORK 



* 

1 



During Stage II of the Peu\ Grant Quality Control Study, QSFA experimented 
with the strategic approach ^oJ$C system development* The methodology proved" 
effective at improving the capacity of operating units in OSFA to measure error and 



to design corrective action*. The major limitation^ of this approach were: ' / 

• * OSFA had limited capability "to identify error-prone -points* in the 

delivery system, since, there was not a comprehensive QC framework; 

• Onve an area was selected for QC development, the actual strategies for 
developing the measurement system had to be identified, „ 

\ ' - .. 



y 



Based on the experience during Stage n, it & now possible to suggest the basic 
elements of such an analysis system. Two parts of t£e Stage II study can be qs^d to 
suggest such a^ framework. ^.First, the framework developed in the Deliver* System- 
Assessment Task, Part II of Stage n, provides a reference point and methodology for 
developing a comprehensive QCMIS framework, one -that can be used to identify QC 
measures^ for key^activities. Wcond, the experience with targets of opportunity 
during the Part I study can be used to help identify, strategies for. developing these 
QC measures. ' s ' " \ ' / 

A framework for developing a comprehensive QCMIS for the OSFA delivery 
system is proposed jn this section. First, however, it is nebessary ^o review the 
framework used in the Delivery System Assessment Task. * 

Delivery System Assessment Framework 

The Delivery System Assessment Task used a very detailed methodology to 
specify the current student aid delivery, system, to develop a model to measure its 
effects^ and to apply this mctd& to the current system. This same methodology can 
be adapted to develop a QC measurement framework. * 

The methodology for developing the analytic model for the Delivery System 
Task required taking a series of analytic steps. These were: 
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• Sjep Is Specify the current delivery system in the form of Input- 
process-output (IPO) chains. 

• St'ep 2s Develop independently a* detailed list of program features for 
each program. * * 

• Step 3s Determine which program featflres influence each delivery 
* system activity. , • . ' t • 

t 4 ' . 

Step M, Determine the intervening variables that are relevant to each 
delivery system activity. 

v 

• Step 5: Determine which effects are influenced by each delivery system 
^ activity, _ 

• Step 6s Develop ^measures for each effect at each delivery system 
activity. . * 

J » m " Step 7; Identify existing data sources or '.develop new data sources for 
each measure. ' L . 

• Step & Identify methods of analysis for each effect at each system step. 



This process resulted in a preliminary specification of the entire student aid 
delivery system. This specification is currently being revised. The refined list of 
delivery system activities is contained in Figure 3-6 in Chapter 3. A series of 
reports were generated during the course of this analysis. These were: 

• * * * ' 

• A review of previous approaches to delivery system., issues, which 
resulted in a context paper; 

i 

» • A preliminary model, which^provided the logic for developing the 
\ \ detailed model^ 

• A specification of the current system, including program features (from 
laws, regulatjpns, and administrative decisions) and system steps (in the 
form of input, process, output chains) for each activity in the delivery 
.system? , • 

- ■ - * ' 

• , ,' t A general assessment model, which identifies intervening variables and 

* effects for each delivery system activity? 

k, An analytic agenda, which identifies* measures, data sources, and 
methods of analysis for each effect. 




The three key reports in this series, as far 'as the methodology for the QC 
measurement framework is concerned, are the specifications, the general model, 
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and the" analytic agenda. The relationship between the three documents is 

\, Illustrated/ ip Figure^ A^7. The. specification document identified the program 

features ,antf delivery system steps associated with eacl? activity in the current 
delivery system. ' 

m , 

The General Assessment Model is being applied currentlyjn the. analysis of the 
>* effects of the current delivery system alternatives. The steps, required* to apply the 
model ares ' * .• . < 

'■ ' \ • • * .• . - ■) .' ' ' 

• Evaluate the current system, including an estimation of baseline.eftects 
on all participants; 

^ • Identify delivery system alternatives, based on review of past proposals 
and community input; 




Specify, selected alternatives, including program features and system 
steps; 

• .. 

• Assess alternatives, including estimation of differential effects on 
participants; 

• Specify intent, perhaps in different ways; t 

f ' , ' ' *• 

• Rank alternatives according^© specification of intent. 

« 

* • , '.,■■'-*• 

If the QC development framework is developed, then it can be systematically 
applied, using the strategic approach, to high error-prone points in the delivery 
system. In fact, the results of the assessment of the current system can be used to 
identify error-prone points. 

Delivery Level QC Measurement Framework 

The framework and analysis completed*^ as , part -of the Delivery System 
Assessment Task provide the basis for the proposed framework for the delivery level 
QC measurement framework. There are two key steps in the development jA the 
framework. . 

First, the specifications ^cumeM provides a starting point for specifying the 
types of quality control that cat? be developed for each critical activity. Figure A-S 
illustrates the framework lor doing^this. The specification, of the current' system 
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*A preliminary specification for the current student aid delivery system was developed by Advanced Technology, Inc., as part of the 
Delivery System .Assessment Task. This specification is currently being revised to* incorporate input from OSFA personnel. 
~ : ■ ' ' 'i 

The format for this report will be similar to the "Analytic Agenda to the Current System," developed as part of, the Delivery System 
Assessment Task. The content will reflect the findings of that analysis. 
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identifies the ^program features for each, activity, as well as the "delivery systems 

* required to complete each activity, in the form of inputs, processes, and outputs. It 
is possible to use those specifications to develop the QC framework. Specifically, 

• for each critical activity, it would be necessary to: 

, • Identify the types of quality control that apply, including receipt control, 
productivity control, and fund control; 

\ - 

• Identify the specific QC measures that apply such as timing of receipt of 
documents and completion of reviews for production control*, 

• Identiiy potential data sources for all measures; 

* \ . ' 

• Identify the specific strategies that could be used to develop the QC 
measurement component; 

For illustrative purposes, these steps have to be, applied to two delivery system 
activities. Figure A-9 applies it to activity 2; student application examples 
illustrates how the vendor/processor" function for Pell would be iiiustated in this 
framework. The second illustration is Figure A-iO,. activity. 6.6, GSi claims and 
collections, which is kn activity that is currently being reviewed for QC 

, The second key element of the framework will be the reanaiysis of the results 
of the assessment of the current delivery system, which has been recently 
completed. The assessment of 'the current system provides baseline measures for 
each delivery system effect and analyzes how individual activities contribute to 
each effect. A reanaiysis of this data could help inform the QC framework in two 
ways. These ares 

• " The analysis would be used to identify the measurement strategies for 

each activity; 

f . ' 

• It could also be used to Identify error-prone points in the delivery 
system. 

* 

Both oi these features are critical to the QC measurement framework. One of 
the major obstacles impeding the widespread application of the strategic #&roach is 
that the specific factors that should be measured—the types of QC that apply, the 
possible measures, and the strategies for developing the measures— must - be 
developed on an ad hoc basis. This framework, building on the experience gained in 

i 
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. FIGURE A-9 (Continued) 
EXAMPLE #1 OF DELIVERY LEVEL (QCMIS) FRAMEWORK: STUDENT 
APPLICATION ACTIVITY (FOR AN APPLICATION PROCESSING CONTRACT) 
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CLAIMS AMD COLLECTIONS ACTIVITY 
EXAMPLE #2 OF DELIVERY LEVEL (QCMIS) FRAMEWORK 
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FIGURE A- 10 (Cont.) 
CLAIMS AND> COLLECTIONS ACTIVITY 
EXAMPLE #2 OF DELIVERY LEVEL (QCMIS) FRAMEWORK 
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FIGURE A- 10 (ConU ^ 
CLAIMS AND COLLECTIONS ACTIVITY 
EXAMPLE #2 OF DELIVERY LEVEL (QCMIS) FRAMEWORK 
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the earlier . analysis would identify these factors- This identification of 
measurement strategies could be informed by the analysis that has already taken 
place. 

Additionally, 'the reanaiysis of the evaluation of the current system woufo" 
identify error-prone points in the delivery system, this analysis could focus on 
identifying the activities in the delivery system that cause the most problems to tne 
Federal government, states, institutions, and applicants. 4} 

- , - f . 

In summary, the near completion of the Delivery 'System Assessment Task* 
provides OSFA with .an excellent opportunity to develop a solid and comprehensive 
QC framework for the sfudent aid delivery system. The results of this analysis 
provide a starting point, methodology, and data base for such an exercise. v * 
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^ IW* chart tlvti the activities fev ^iJi-i)r»Um whtcMte currently utilized to Implement the PeR Grant* CSt and Compos -Rased programs. Tht fn*pott of fhti chart 
1» t«o*toM, It act* « * t*Me of contents lor tfte jpecUtcaUdo chart vand H note* acttvttfe* whlc*h are currently Integrated to some degree across the three program 
component*. Activities which a*e at frail partliUy Integrated across the three programs ate marked with an asterisk C P |L „ . ' ■ 
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* This activity Is Identical and completely Integrated for the fell and Campus-Bated component*. 
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